Showing 1–9 of 9 results for author: Ni, P

MPTF-Net: Multi-view Pyramid Transformer Fusion Network for LiDAR-based Place Recognition

Authors: Shuyuan Li, Zihang Wang, Xieyuanli Chen, Wenkai Zhu, Xiaoteng Fang, Peizhou Ni, Junhao Yang, Dong Kong

Abstract: LiDAR-based place recognition (LPR) is essential for global localization and loop-closure detection in large-scale SLAM systems. Existing methods typically construct global descriptors from Range Images or BEV representations for matching. BEV is widely adopted due to its explicit 2D spatial layout encoding and efficient retrieval. However, conventional BEV representations rely on simple statistic… ▽ More LiDAR-based place recognition (LPR) is essential for global localization and loop-closure detection in large-scale SLAM systems. Existing methods typically construct global descriptors from Range Images or BEV representations for matching. BEV is widely adopted due to its explicit 2D spatial layout encoding and efficient retrieval. However, conventional BEV representations rely on simple statistical aggregation, which fails to capture fine-grained geometric structures, leading to performance degradation in complex or repetitive environments. To address this, we propose MPTF-Net, a novel multi-view multi-scale pyramid Transformer fusion network. Our core contribution is a multi-channel NDT-based BEV encoding that explicitly models local geometric complexity and intensity distributions via Normal Distribution Transform, providing a noise-resilient structural prior. To effectively integrate these features, we develop a customized pyramid Transformer module that captures cross-view interactive correlations between Range Image Views (RIV) and NDT-BEV at multiple spatial scales. Extensive experiments on the nuScenes, KITTI and NCLT datasets demonstrate that MPTF-Net achieves state-of-the-art performance, specifically attaining a Recall@1 of 96.31\% on the nuScenes Boston split while maintaining an inference latency of only 10.02 ms, making it highly suitable for real-time autonomous unmanned systems. △ Less

Submitted 6 April, 2026; originally announced April 2026.

Reasoning and Learning a Perceptual Metric for Self-Training of Reflective Objects in Bin-Picking with a Low-cost Camera

Authors: Peiyuan Ni, Chee Meng Chew, Marcelo H. Ang Jr., Gregory S. Chirikjian

Abstract: Bin-picking of metal objects using low-cost RGB-D cameras often suffers from sparse depth information and reflective surface textures, leading to errors and the need for manual labeling. To reduce human intervention, we propose a two-stage framework consisting of a metric learning stage and a self-training stage. Specifically, to automatically process data captured by a low-cost camera (LC), we in… ▽ More Bin-picking of metal objects using low-cost RGB-D cameras often suffers from sparse depth information and reflective surface textures, leading to errors and the need for manual labeling. To reduce human intervention, we propose a two-stage framework consisting of a metric learning stage and a self-training stage. Specifically, to automatically process data captured by a low-cost camera (LC), we introduce a Multi-object Pose Reasoning (MoPR) algorithm that optimizes pose hypotheses under depth, collision, and boundary constraints. To further refine pose candidates, we adopt a Symmetry-aware Lie-group based Bayesian Gaussian Mixture Model (SaL-BGMM), integrated with the Expectation-Maximization (EM) algorithm, for symmetry-aware filtering. Additionally, we propose a Weighted Ranking Information Noise Contrastive Estimation (WR-InfoNCE) loss to enable the LC to learn a perceptual metric from reconstructed data, supporting self-training on untrained or even unseen objects. Experimental results show that our approach outperforms several state-of-the-art methods on both the ROBI dataset and our newly introduced Self-ROBI dataset. △ Less

Submitted 16 August, 2025; v1 submitted 26 March, 2025; originally announced March 2025.

Comments: 8 pages, 10 figures; Accepted by IEEE RAL, presentation at ICRA 2026

Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine

Authors: Qiao Jin, Fangyuan Chen, Yiliang Zhou, Ziyang Xu, Justin M. Cheung, Robert Chen, Ronald M. Summers, Justin F. Rousseau, Peiyun Ni, Marc J Landsman, Sally L. Baxter, Subhi J. Al'Aref, Yijia Li, Alex Chen, Josef A. Brejt, Michael F. Chiang, Yifan Peng, Zhiyong Lu

Abstract: Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V's rationales of image comprehension, recall of medical knowledge, and step-by… ▽ More Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V's rationales of image comprehension, recall of medical knowledge, and step-by-step multimodal reasoning when solving New England Journal of Medicine (NEJM) Image Challenges - an imaging quiz designed to test the knowledge and diagnostic capabilities of medical professionals. Evaluation results confirmed that GPT-4V performs comparatively to human physicians regarding multi-choice accuracy (81.6% vs. 77.8%). GPT-4V also performs well in cases where physicians incorrectly answer, with over 78% accuracy. However, we discovered that GPT-4V frequently presents flawed rationales in cases where it makes the correct final choices (35.5%), most prominent in image comprehension (27.2%). Regardless of GPT-4V's high accuracy in multi-choice questions, our findings emphasize the necessity for further in-depth evaluations of its rationales before integrating such multimodal AI models into clinical workflows. △ Less

Submitted 31 August, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Journal ref: npj Digital Medicine, 2024

Continual Graph Learning: A Survey

Authors: Qiao Yuan, Sheng-Uei Guan, Pin Ni, Tianlun Luo, Ka Lok Man, Prudence Wong, Victor Chang

Abstract: Continual Graph Learning (CGL) enables models to incrementally learn from streaming graph-structured data without forgetting previously acquired knowledge. Experience replay is a common solution that reuses a subset of past samples during training. However, it may lead to information loss and privacy risks. Generative replay addresses these concerns by synthesizing informative subgraphs for rehear… ▽ More Continual Graph Learning (CGL) enables models to incrementally learn from streaming graph-structured data without forgetting previously acquired knowledge. Experience replay is a common solution that reuses a subset of past samples during training. However, it may lead to information loss and privacy risks. Generative replay addresses these concerns by synthesizing informative subgraphs for rehearsal. Existing generative replay approaches often rely on graph condensation via distribution matching, which faces two key challenges: (1) the use of random feature encodings may fail to capture the characteristic kernel of the discrepancy metric, weakening distribution alignment; and (2) matching over a fixed small subgraph cannot guarantee low risk on previous tasks, as indicated by domain adaptation theory. To overcome these limitations, we propose an Adversarial Condensation based Generative Replay (ACGR) framwork. It reformulates graph condensation as a min-max optimization problem to achieve better distribution matching. Moreover, instead of learning a single subgraph, we learn its distribution, allowing for the generation of multiple samples and improved empirical risk minimization. Experiments on three benchmark datasets demonstrate that ACGR outperforms existing methods in both accuracy and stability. △ Less

Submitted 29 March, 2026; v1 submitted 28 January, 2023; originally announced January 2023.

Journal ref: Pattern Recognition, 113600 (2026)

Attention-based Ingredient Phrase Parser

Authors: Zhengxiang Shi, Pin Ni, Meihui Wang, To Eun Kim, Aldo Lipani

Abstract: As virtual personal assistants have now penetrated the consumer market, with products such as Siri and Alexa, the research community has produced several works on task-oriented dialogue tasks such as hotel booking, restaurant booking, and movie recommendation. Assisting users to cook is one of these tasks that are expected to be solved by intelligent assistants, where ingredients and their corresp… ▽ More As virtual personal assistants have now penetrated the consumer market, with products such as Siri and Alexa, the research community has produced several works on task-oriented dialogue tasks such as hotel booking, restaurant booking, and movie recommendation. Assisting users to cook is one of these tasks that are expected to be solved by intelligent assistants, where ingredients and their corresponding attributes, such as name, unit, and quantity, should be provided to users precisely and promptly. However, existing ingredient information scraped from the cooking website is in the unstructured form with huge variation in the lexical structure, for example, '1 garlic clove, crushed', and '1 (8 ounce) package cream cheese, softened', making it difficult to extract information exactly. To provide an engaged and successful conversational service to users for cooking tasks, we propose a new ingredient parsing model that can parse an ingredient phrase of recipes into the structure form with its corresponding attributes with over 0.93 F1-score. Experimental results show that our model achieves state-of-the-art performance on AllRecipes and Food.com datasets. △ Less

Submitted 5 October, 2022; originally announced October 2022.

Comments: ESANN 2022 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning

Deep learning on rail profiles matching

Authors: Kunqi Wang, Daolin Si, Pu Wang, Jing Ge, Peiyuan Ni, Shuguo Wang

Abstract: Matching the rail cross-section profiles measured on site with the designed profile is a must to evaluate the wear of the rail, which is very important for track maintenance and rail safety. So far, the measured rail profiles to be matched usually have four features, that is, large amount of data, diverse section shapes, hardware made errors, and human experience needs to be introduced to solve th… ▽ More Matching the rail cross-section profiles measured on site with the designed profile is a must to evaluate the wear of the rail, which is very important for track maintenance and rail safety. So far, the measured rail profiles to be matched usually have four features, that is, large amount of data, diverse section shapes, hardware made errors, and human experience needs to be introduced to solve the complex situation on site during matching process. However, traditional matching methods based on feature points or feature lines could no longer meet the requirements. To this end, we first establish the rail profiles matching dataset composed of 46386 pairs of professional manual matched data, then propose a general high-precision method for rail profiles matching using pre-trained convolutional neural network (CNN). This new method based on deep learning is promising to be the dominant approach for this issue. Source code is at https://github.com/Kunqi1994/Deep-learning-on-rail-profile-matching. △ Less

Submitted 29 July, 2022; v1 submitted 17 May, 2022; originally announced May 2022.

An Accelerator for Rule Induction in Fuzzy Rough Theory

Authors: Suyun Zhao, Zhigang Dai, Xizhao Wang, Peng Ni, Hengheng Luo, Hong Chen, Cuiping Li

Abstract: Rule-based classifier, that extract a subset of induced rules to efficiently learn/mine while preserving the discernibility information, plays a crucial role in human-explainable artificial intelligence. However, in this era of big data, rule induction on the whole datasets is computationally intensive. So far, to the best of our knowledge, no known method focusing on accelerating rule induction h… ▽ More Rule-based classifier, that extract a subset of induced rules to efficiently learn/mine while preserving the discernibility information, plays a crucial role in human-explainable artificial intelligence. However, in this era of big data, rule induction on the whole datasets is computationally intensive. So far, to the best of our knowledge, no known method focusing on accelerating rule induction has been reported. This is first study to consider the acceleration technique to reduce the scale of computation in rule induction. We propose an accelerator for rule induction based on fuzzy rough theory; the accelerator can avoid redundant computation and accelerate the building of a rule classifier. First, a rule induction method based on consistence degree, called Consistence-based Value Reduction (CVR), is proposed and used as basis to accelerate. Second, we introduce a compacted search space termed Key Set, which only contains the key instances required to update the induced rule, to conduct value reduction. The monotonicity of Key Set ensures the feasibility of our accelerator. Third, a rule-induction accelerator is designed based on Key Set, and it is theoretically guaranteed to display the same results as the unaccelerated version. Specifically, the rank preservation property of Key Set ensures consistency between the rule induction achieved by the accelerator and the unaccelerated method. Finally, extensive experiments demonstrate that the proposed accelerator can perform remarkably faster than the unaccelerated rule-based classifier methods, especially on datasets with numerous instances. △ Less

Submitted 7 January, 2022; originally announced January 2022.

Comments: 15 pages,9 figures

PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Clouds

Authors: Peiyuan Ni, Wenguang Zhang, Xiaoxiao Zhu, Qixin Cao

Abstract: Grasping for novel objects is important for robot manipulation in unstructured environments. Most of current works require a grasp sampling process to obtain grasp candidates, combined with local feature extractor using deep learning. This pipeline is time-costly, expecially when grasp points are sparse such as at the edge of a bowl. In this paper, we propose an end-to-end approach to directly pre… ▽ More Grasping for novel objects is important for robot manipulation in unstructured environments. Most of current works require a grasp sampling process to obtain grasp candidates, combined with local feature extractor using deep learning. This pipeline is time-costly, expecially when grasp points are sparse such as at the edge of a bowl. In this paper, we propose an end-to-end approach to directly predict the poses, categories and scores (qualities) of all the grasps. It takes the whole sparse point clouds as the input and requires no sampling or search process. Moreover, to generate training data of multi-object scene, we propose a fast multi-object grasp detection algorithm based on Ferrari Canny metrics. A single-object dataset (79 objects from YCB object set, 23.7k grasps) and a multi-object dataset (20k point clouds with annotations and masks) are generated. A PointNet++ based network combined with multi-mask loss is introduced to deal with different training points. The whole weight size of our network is only about 11.6M, which takes about 102ms for a whole prediction process using a GeForce 840M GPU. Our experiment shows our work get 71.43% success rate and 91.60% completion rate, which performs better than current state-of-art works. △ Less

Submitted 21 March, 2020; originally announced March 2020.

Comments: Accepted at the International Conference on Robotics and Automation (ICRA) 2020

A Fast Compressive Sensing Based Digital Image Encryption Technique using Structurally Random Matrices and Arnold Transform

Authors: Nitin Rawat, Pavel Ni, Rajesh Kumar

Abstract: A new digital image encryption method based on fast compressed sensing approach using structurally random matrices and Arnold transform is proposed. Considering the natural images to be compressed in any domain, the fast compressed sensing based approach saves computational time, increases the quality of the image and reduces the dimension of the digital image by choosing even 25 % of the measurem… ▽ More A new digital image encryption method based on fast compressed sensing approach using structurally random matrices and Arnold transform is proposed. Considering the natural images to be compressed in any domain, the fast compressed sensing based approach saves computational time, increases the quality of the image and reduces the dimension of the digital image by choosing even 25 % of the measurements. First, dimension reduction is utilized to compress the digital image with scrambling effect. Second, Arnold transformation is used to give the reduced digital image into more complex form. Then, the complex image is again encrypted by double random phase encoding process embedded with a host image; two random keys with fractional Fourier transform are been used as a secret keys. At the receiver, the decryption process is recovered by using TwIST algorithm. Experimental results including peak-to-peak signal-to-noise ratio between the original and reconstructed image are shown to analyze the validity of this technique and demonstrated our proposed method to be secure, fast, complex and robust. △ Less

Submitted 13 February, 2014; originally announced February 2014.

Comments: 13 pages