-
Molecular dynamics with a first-principles-validated universal machine-learning potential reveals dynamic elementary processes of growth-related adspecies on GaN(0001)
Authors:
Yoshito Takaesu,
Akira Kusaba,
Junko Ishii,
Shigenori Matsushima,
Yoshihiro Kangawa
Abstract:
Atomic-scale understanding of the surface elementary processes in metalorganic vapor phase epitaxy (MOVPE) of GaN has so far relied on static density-functional-theory (DFT) energetics and on first-principles molecular dynamics (FPMD) limited to a few tens of picoseconds. Here we combine FPMD with a universal machine-learning interatomic potential (MLIP), UMA, to follow the dynamics of growth-rela…
▽ More
Atomic-scale understanding of the surface elementary processes in metalorganic vapor phase epitaxy (MOVPE) of GaN has so far relied on static density-functional-theory (DFT) energetics and on first-principles molecular dynamics (FPMD) limited to a few tens of picoseconds. Here we combine FPMD with a universal machine-learning interatomic potential (MLIP), UMA, to follow the dynamics of growth-related adspecies on GaN(0001) over time scales inaccessible to purely first-principles approaches. FPMD simulations of a GaNH admolecule coexisting with H adatoms reveal a hitherto unrecognized diffusion mode, in which the N atom lifts the Ga atom of the GaNH unit off the surface layer during migration, and show that the lifted Ga abstracts an H adatom from the surface, events invisible to static DFT. Single-point UMA calculations on FPMD snapshots reproduce the first-principles relative energies along the trajectory (RMSE of about 8.5 meV/atom) without any retraining. Long-time MLIP-based MD (150 ps) then reveals dynamics never observed within the FPMD window: site-to-site H-adatom hopping, which gates the migration paths of the growth unit, and reversible dissociation of the GaNH unit into independently migrating Ga and NH adspecies. This work constitutes, to our knowledge, the first application of an MLIP to the molecular dynamics of GaN MOVPE.
△ Less
Submitted 26 July, 2026;
originally announced July 2026.
-
MDL Meets Latent Confounders: LNML-based Causal Discovery
Authors:
Zhongyi Que,
Shin Matsushima,
Kenji Yamanishi
Abstract:
Causal discovery with nonlinear mechanisms and latent confounders remains challenging. Existing methods often rely on either linear assumptions or causal sufficiency, limiting their applicability. We propose an MDL-based causal discovery framework that explicitly accounts for latent confounders while allowing flexible nonlinear mechanisms by minimizing the luckiness normalized maximum likelihood (…
▽ More
Causal discovery with nonlinear mechanisms and latent confounders remains challenging. Existing methods often rely on either linear assumptions or causal sufficiency, limiting their applicability. We propose an MDL-based causal discovery framework that explicitly accounts for latent confounders while allowing flexible nonlinear mechanisms by minimizing the luckiness normalized maximum likelihood (LNML) code-length. The causal relationship between each variable pair is determined by selecting the shortest code-length of the causal model, and we introduce the notion of $Δ$-pseudo-collinearity to identify dependencies induced by latent confounders. Based on these ideas, we develop a greedy algorithm, termed Pseudo-Collinearity Guided Causal Discovery (PCG-CD). Experiments on synthetic and real-world datasets demonstrate that the proposed method accurately recovers directed causal relationships and effectively detects latent confounders.
△ Less
Submitted 5 July, 2026;
originally announced July 2026.
-
Enhancing and Exploring Mild Cognitive Impairment Detection with W2V-BERT-2.0
Authors:
Yueguan Wang,
Tatsunari Matsushima,
Soichiro Matsushima,
Toshimitsu Sakai
Abstract:
This study explores a multi-lingual audio self-supervised learning model for detecting mild cognitive impairment (MCI) using the TAUKADIAL cross-lingual dataset. While speech transcription-based detection with BERT models is effective, limitations exist due to a lack of transcriptions and temporal information. To address these issues, the study utilizes features directly from speech utterances wit…
▽ More
This study explores a multi-lingual audio self-supervised learning model for detecting mild cognitive impairment (MCI) using the TAUKADIAL cross-lingual dataset. While speech transcription-based detection with BERT models is effective, limitations exist due to a lack of transcriptions and temporal information. To address these issues, the study utilizes features directly from speech utterances with W2V-BERT-2.0. We propose a visualization method to detect essential layers of the model for MCI classification and design a specific inference logic considering the characteristics of MCI. The experiment shows competitive results, and the proposed inference logic significantly contributes to the improvements from the baseline. We also conduct detailed analysis which reveals the challenges related to speaker bias in the features and the sensitivity of MCI classification accuracy to the data split, providing valuable insights for future research.
△ Less
Submitted 27 January, 2025;
originally announced January 2025.
-
Detection of Unobserved Common Causes based on NML Code in Discrete, Mixed, and Continuous Variables
Authors:
Masatoshi Kobayashi,
Kohei Miyagichi,
Shin Matsushima
Abstract:
Causal discovery in the presence of unobserved common causes from observational data only is a crucial but challenging problem. We categorize all possible causal relationships between two random variables into the following four categories and aim to identify one from observed data: two cases in which either of the direct causality exists, a case that variables are independent, and a case that var…
▽ More
Causal discovery in the presence of unobserved common causes from observational data only is a crucial but challenging problem. We categorize all possible causal relationships between two random variables into the following four categories and aim to identify one from observed data: two cases in which either of the direct causality exists, a case that variables are independent, and a case that variables are confounded by latent confounders. Although existing methods have been proposed to tackle this problem, they require unobserved variables to satisfy assumptions on the form of their equation models. In our previous study (Kobayashi et al., 2022), the first causal discovery method without such assumptions is proposed for discrete data and named CLOUD. Using Normalized Maximum Likelihood (NML) Code, CLOUD selects a model that yields the minimum codelength of the observed data from a set of model candidates. This paper extends CLOUD to apply for various data types across discrete, mixed, and continuous. We not only performed theoretical analysis to show the consistency of CLOUD in terms of the model selection, but also demonstrated that CLOUD is more effective than existing methods in inferring causal relationships by extensive experiments on both synthetic and real-world data.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
mdx: A Cloud Platform for Supporting Data Science and Cross-Disciplinary Research Collaborations
Authors:
Toyotaro Suzumura,
Akiyoshi Sugiki,
Hiroyuki Takizawa,
Akira Imakura,
Hiroshi Nakamura,
Kenjiro Taura,
Tomohiro Kudoh,
Toshihiro Hanawa,
Yuji Sekiya,
Hiroki Kobayashi,
Shin Matsushima,
Yohei Kuga,
Ryo Nakamura,
Renhe Jiang,
Junya Kawase,
Masatoshi Hanai,
Hiroshi Miyazaki,
Tsutomu Ishizaki,
Daisuke Shimotoku,
Daisuke Miyamoto,
Kento Aida,
Atsuko Takefusa,
Takashi Kurimoto,
Koji Sasayama,
Naoya Kitagawa
, et al. (8 additional authors not shown)
Abstract:
The growing amount of data and advances in data science have created a need for a new kind of cloud platform that provides users with flexibility, strong security, and the ability to couple with supercomputers and edge devices through high-performance networks. We have built such a nation-wide cloud platform, called "mdx" to meet this need. The mdx platform's virtualization service, jointly operat…
▽ More
The growing amount of data and advances in data science have created a need for a new kind of cloud platform that provides users with flexibility, strong security, and the ability to couple with supercomputers and edge devices through high-performance networks. We have built such a nation-wide cloud platform, called "mdx" to meet this need. The mdx platform's virtualization service, jointly operated by 9 national universities and 2 national research institutes in Japan, launched in 2021, and more features are in development. Currently mdx is used by researchers in a wide variety of domains, including materials informatics, geo-spatial information science, life science, astronomical science, economics, social science, and computer science. This paper provides an the overview of the mdx platform, details the motivation for its development, reports its current status, and outlines its future plans.
△ Less
Submitted 26 March, 2022;
originally announced March 2022.
-
Statistical Learnability of Generalized Additive Models based on Total Variation Regularization
Authors:
Shin Matsushima
Abstract:
A generalized additive model (GAM, Hastie and Tibshirani (1987)) is a nonparametric model by the sum of univariate functions with respect to each explanatory variable, i.e., $f({\mathbf x}) = \sum f_j(x_j)$, where $x_j\in\mathbb{R}$ is $j$-th component of a sample ${\mathbf x}\in \mathbb{R}^p$. In this paper, we introduce the total variation (TV) of a function as a measure of the complexity of fun…
▽ More
A generalized additive model (GAM, Hastie and Tibshirani (1987)) is a nonparametric model by the sum of univariate functions with respect to each explanatory variable, i.e., $f({\mathbf x}) = \sum f_j(x_j)$, where $x_j\in\mathbb{R}$ is $j$-th component of a sample ${\mathbf x}\in \mathbb{R}^p$. In this paper, we introduce the total variation (TV) of a function as a measure of the complexity of functions in $L^1_{\rm c}(\mathbb{R})$-space. Our analysis shows that a GAM based on TV-regularization exhibits a Rademacher complexity of $O(\sqrt{\frac{\log p}{m}})$, which is tight in terms of both $m$ and $p$ in the agnostic case of the classification problem. In result, we obtain generalization error bounds for finite samples according to work by Bartlett and Mandelson (2002).
△ Less
Submitted 16 February, 2018; v1 submitted 8 February, 2018;
originally announced February 2018.
-
Grafting for Combinatorial Boolean Model using Frequent Itemset Mining
Authors:
Taito Lee,
Shin Matsushima,
Kenji Yamanishi
Abstract:
This paper introduces the combinatorial Boolean model (CBM), which is defined as the class of linear combinations of conjunctions of Boolean attributes. This paper addresses the issue of learning CBM from labeled data. CBM is of high knowledge interpretability but naïve learning of it requires exponentially large computation time with respect to data dimension and sample size. To overcome this com…
▽ More
This paper introduces the combinatorial Boolean model (CBM), which is defined as the class of linear combinations of conjunctions of Boolean attributes. This paper addresses the issue of learning CBM from labeled data. CBM is of high knowledge interpretability but naïve learning of it requires exponentially large computation time with respect to data dimension and sample size. To overcome this computational difficulty, we propose an algorithm GRAB (GRAfting for Boolean datasets), which efficiently learns CBM within the $L_1$-regularized loss minimization framework. The key idea of GRAB is to reduce the loss minimization problem to the weighted frequent itemset mining, in which frequent patterns are efficiently computable. We employ benchmark datasets to empirically demonstrate that GRAB is effective in terms of computational efficiency, prediction accuracy and knowledge discovery.
△ Less
Submitted 13 November, 2017; v1 submitted 7 November, 2017;
originally announced November 2017.
-
DS-MLR: Exploiting Double Separability for Scaling up Distributed Multinomial Logistic Regression
Authors:
Parameswaran Raman,
Sriram Srinivasan,
Shin Matsushima,
Xinhua Zhang,
Hyokun Yun,
S. V. N. Vishwanathan
Abstract:
Scaling multinomial logistic regression to datasets with very large number of data points and classes is challenging. This is primarily because one needs to compute the log-partition function on every data point. This makes distributing the computation hard. In this paper, we present a distributed stochastic gradient descent based optimization method (DS-MLR) for scaling up multinomial logistic re…
▽ More
Scaling multinomial logistic regression to datasets with very large number of data points and classes is challenging. This is primarily because one needs to compute the log-partition function on every data point. This makes distributing the computation hard. In this paper, we present a distributed stochastic gradient descent based optimization method (DS-MLR) for scaling up multinomial logistic regression problems to massive scale datasets without hitting any storage constraints on the data and model parameters. Our algorithm exploits double-separability, an attractive property that allows us to achieve both data as well as model parallelism simultaneously. In addition, we introduce a non-blocking and asynchronous variant of our algorithm that avoids bulk-synchronization. We demonstrate the versatility of DS-MLR to various scenarios in data and model parallelism, through an extensive empirical study using several real-world datasets. In particular, we demonstrate the scalability of DS-MLR by solving an extreme multi-class classification problem on the Reddit dataset (159 GB data, 358 GB parameters) where, to the best of our knowledge, no other existing methods apply.
△ Less
Submitted 3 August, 2018; v1 submitted 16 April, 2016;
originally announced April 2016.
-
WordRank: Learning Word Embeddings via Robust Ranking
Authors:
Shihao Ji,
Hyokun Yun,
Pinar Yanardag,
Shin Matsushima,
S. V. N. Vishwanathan
Abstract:
Embedding words in a vector space has gained a lot of attention in recent years. While state-of-the-art methods provide efficient computation of word similarities via a low-dimensional matrix embedding, their motivation is often left unclear. In this paper, we argue that word embedding can be naturally viewed as a ranking problem due to the ranking nature of the evaluation metrics. Then, based on…
▽ More
Embedding words in a vector space has gained a lot of attention in recent years. While state-of-the-art methods provide efficient computation of word similarities via a low-dimensional matrix embedding, their motivation is often left unclear. In this paper, we argue that word embedding can be naturally viewed as a ranking problem due to the ranking nature of the evaluation metrics. Then, based on this insight, we propose a novel framework WordRank that efficiently estimates word representations via robust ranking, in which the attention mechanism and robustness to noise are readily achieved via the DCG-like ranking losses. The performance of WordRank is measured in word similarity and word analogy benchmarks, and the results are compared to the state-of-the-art word embedding techniques. Our algorithm is very competitive to the state-of-the- arts on large corpora, while outperforms them by a significant margin when the training set is limited (i.e., sparse and noisy). With 17 million tokens, WordRank performs almost as well as existing methods using 7.2 billion tokens on a popular word similarity benchmark. Our multi-node distributed implementation of WordRank is publicly available for general usage.
△ Less
Submitted 27 September, 2016; v1 submitted 8 June, 2015;
originally announced June 2015.
-
Totally Corrective Boosting with Cardinality Penalization
Authors:
Vasil S. Denchev,
Nan Ding,
Shin Matsushima,
S. V. N. Vishwanathan,
Hartmut Neven
Abstract:
We propose a totally corrective boosting algorithm with explicit cardinality regularization. The resulting combinatorial optimization problems are not known to be efficiently solvable with existing classical methods, but emerging quantum optimization technology gives hope for achieving sparser models in practice. In order to demonstrate the utility of our algorithm, we use a distributed classical…
▽ More
We propose a totally corrective boosting algorithm with explicit cardinality regularization. The resulting combinatorial optimization problems are not known to be efficiently solvable with existing classical methods, but emerging quantum optimization technology gives hope for achieving sparser models in practice. In order to demonstrate the utility of our algorithm, we use a distributed classical heuristic optimizer as a stand-in for quantum hardware. Even though this evaluation methodology incurs large time and resource costs on classical computing machinery, it allows us to gauge the potential gains in generalization performance and sparsity of the resulting boosted ensembles. Our experimental results on public data sets commonly used for benchmarking of boosting algorithms decidedly demonstrate the existence of such advantages. If actual quantum optimization were to be used with this algorithm in the future, we would expect equivalent or superior results at much smaller time and energy costs during training. Moreover, studying cardinality-penalized boosting also sheds light on why unregularized boosting algorithms with early stopping often yield better results than their counterparts with explicit convex regularization: Early stopping performs suboptimal cardinality regularization. The results that we present here indicate it is beneficial to explicitly solve the combinatorial problem still left open at early termination.
△ Less
Submitted 6 April, 2015;
originally announced April 2015.
-
Distributed Stochastic Optimization of the Regularized Risk
Authors:
Shin Matsushima,
Hyokun Yun,
Xinhua Zhang,
S. V. N. Vishwanathan
Abstract:
Many machine learning algorithms minimize a regularized risk, and stochastic optimization is widely used for this task. When working with massive data, it is desirable to perform stochastic optimization in parallel. Unfortunately, many existing stochastic optimization algorithms cannot be parallelized efficiently. In this paper we show that one can rewrite the regularized risk minimization problem…
▽ More
Many machine learning algorithms minimize a regularized risk, and stochastic optimization is widely used for this task. When working with massive data, it is desirable to perform stochastic optimization in parallel. Unfortunately, many existing stochastic optimization algorithms cannot be parallelized efficiently. In this paper we show that one can rewrite the regularized risk minimization problem as an equivalent saddle-point problem, and propose an efficient distributed stochastic optimization (DSO) algorithm. We prove the algorithm's rate of convergence; remarkably, our analysis shows that the algorithm scales almost linearly with the number of processors. We also verify with empirical evaluations that the proposed algorithm is competitive with other parallel, general purpose stochastic and batch optimization algorithms for regularized risk minimization.
△ Less
Submitted 9 June, 2015; v1 submitted 17 June, 2014;
originally announced June 2014.