-
L3Cube-IndicQuest v2: A Large-Scale Multilingual Benchmark for Evaluating Factual Knowledge of Large Language Models Across Indic Languages
Authors:
Rinit Jain,
Tirthraj Mahajan,
Advait Joshi,
Raviraj Joshi
Abstract:
We present L3Cube-IndicQuest v2, a large-scale gold-standard multilingual question-answering benchmark for evaluating the India-specific factual knowledge of Large Language Models (LLMs). The benchmark comprises 3,471 curriculum-grounded English question--answer pairs spanning nine domains, curated from educational curricula, competitive examination materials, and domain-specific reference books.…
▽ More
We present L3Cube-IndicQuest v2, a large-scale gold-standard multilingual question-answering benchmark for evaluating the India-specific factual knowledge of Large Language Models (LLMs). The benchmark comprises 3,471 curriculum-grounded English question--answer pairs spanning nine domains, curated from educational curricula, competitive examination materials, and domain-specific reference books. We introduce a practical hybrid construction strategy that combines context-grounded LLM-based question generation and validation with semantic deduplication and human verification, enabling scalable creation of benchmark data while preserving annotation quality. The benchmark is translated into 19 Indic languages, yielding a publicly released multilingual dataset of 69,420 question--answer pairs across 20 languages. We evaluate six LLMs under three protocols: LLM-as-a-judge and two deterministic lexical criteria, exact-substring and word-overlap matching. All three produce almost the same model ranking, showing that the results do not depend on the choice of judge. The frontier commercial model leads by a wide margin, and among open-weight models Gemma4 31B outperforms the Indic-specialised Sarvam 30B in every evaluated Indic language.
△ Less
Submitted 16 August, 2026;
originally announced August 2026.
-
Enhanced Sign Language Translation between American Sign Language (ASL) and Indian Sign Language (ISL) Using LLMs
Authors:
Malay Kumar,
S. Sarvajit Visagan,
Tanish Sarang Mahajan,
Anisha Natarajan
Abstract:
We have come up with a research that hopes to provide a bridge between the users of American Sign Language and the users of spoken language and Indian Sign Language (ISL). The research enabled us to create a novel framework that we have developed for Learner Systems. Leveraging art of Large models to create key features including: - Real-time translation between these two sign languages in an effi…
▽ More
We have come up with a research that hopes to provide a bridge between the users of American Sign Language and the users of spoken language and Indian Sign Language (ISL). The research enabled us to create a novel framework that we have developed for Learner Systems. Leveraging art of Large models to create key features including: - Real-time translation between these two sign languages in an efficient manner. Making LLM's capability available for seamless translations to ISL. Here is the full study showing its implementation in this paper. The core of the system is a sophisticated pipeline that begins with reclassification and recognition of ASL gestures based on a strong Random Forest Classifier. By recognizing the ASL, it is translated into text which can be more easily processed. Highly evolved natural language NLP (Natural Language Processing) techniques come in handy as they play a role in our LLM integration where you then use LLMs to be able to convert the ASL text to ISL which provides you with the intent of sentence or phrase. The final step is to synthesize the translated text back into ISL gestures, creating an end-to-end translation experience using RIFE-Net. This framework is tasked with key challenges such as automatically dealing with gesture variability and overcoming the linguistic differences between ASL and ISL. By automating the translation process, we hope to vastly improve accessibility for sign language users. No longer will the communication gap between ASL and ISL create barriers; this totally cool innovation aims to bring our communities closer together. And we believe, with full confidence in our framework, that we're able to apply the same principles across a wide variety of sign language dialects.
△ Less
Submitted 19 November, 2024;
originally announced November 2024.
-
Solid State Detectors and Tracking for Snowmass
Authors:
A. Affolder,
A. Apresyan,
S. Worm,
M. Albrow,
D. Ally,
D. Ambrose,
E. Anderssen,
N. Apadula,
P. Asenov,
W. Armstrong,
M. Artuso,
A. Barbier,
P. Barletta,
L. Bauerdick,
D. Berry,
M. Bomben,
M. Boscardin,
J. Brau,
W. Brooks,
M. Breidenbach,
J. Buckley,
V. Cairo,
R. Caputo,
L. Carpenter,
M. Centis-Vignali
, et al. (110 additional authors not shown)
Abstract:
Tracking detectors are of vital importance for collider-based high energy physics (HEP) experiments. The primary purpose of tracking detectors is the precise reconstruction of charged particle trajectories and the reconstruction of secondary vertices. The performance requirements from the community posed by the future collider experiments require an evolution of tracking systems, necessitating the…
▽ More
Tracking detectors are of vital importance for collider-based high energy physics (HEP) experiments. The primary purpose of tracking detectors is the precise reconstruction of charged particle trajectories and the reconstruction of secondary vertices. The performance requirements from the community posed by the future collider experiments require an evolution of tracking systems, necessitating the development of new techniques, materials and technologies in order to fully exploit their physics potential. In this article we summarize the discussions and conclusions of the 2022 Snowmass Instrumentation Frontier subgroup on Solid State and Tracking Detectors (Snowmass IF03).
△ Less
Submitted 19 October, 2022; v1 submitted 8 September, 2022;
originally announced September 2022.
-
Novel Sensors for Particle Tracking: a Contribution to the Snowmass Community Planning Exercise of 2021
Authors:
M. R. Hoeferkamp,
S. Seidel,
S. Kim,
J. Metcalfe,
A. Sumant,
H. Kagan,
W. Trischuk,
M. Boscardin,
G. -F. Dalla Betta,
D. M. S. Sultan,
N. T. Fourches,
C. Renard,
A. Barbier,
T. Mahajan,
A. Minns,
V. Tokranov,
M. Yakimov,
S. Oktyabrsky,
C. Gingu,
P. Murat,
M. T. Hedges
Abstract:
Five contemporary technologies are discussed in the context of their potential roles in particle tracking for future high energy physics applications. These include sensors of the 3D configuration, in both diamond and silicon, submicron-dimension pixels, thin film detectors, and scintillating quantum dots in gallium arsenide. Drivers of the technologies include radiation hardness, excellent positi…
▽ More
Five contemporary technologies are discussed in the context of their potential roles in particle tracking for future high energy physics applications. These include sensors of the 3D configuration, in both diamond and silicon, submicron-dimension pixels, thin film detectors, and scintillating quantum dots in gallium arsenide. Drivers of the technologies include radiation hardness, excellent position, vertex, and timing resolution, simplified integration, and optimized power, cost, and material.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Nowcasting of COVID-19 confirmed cases: Foundations, trends, and challenges
Authors:
Tanujit Chakraborty,
Indrajit Ghosh,
Tirna Mahajan,
Tejasvi Arora
Abstract:
The coronavirus disease 2019 (COVID-19) has become a public health emergency of international concern affecting more than 200 countries and territories worldwide. As of September 30, 2020, it has caused a pandemic outbreak with more than 33 million confirmed infections and more than 1 million reported deaths worldwide. Several statistical, machine learning, and hybrid models have previously tried…
▽ More
The coronavirus disease 2019 (COVID-19) has become a public health emergency of international concern affecting more than 200 countries and territories worldwide. As of September 30, 2020, it has caused a pandemic outbreak with more than 33 million confirmed infections and more than 1 million reported deaths worldwide. Several statistical, machine learning, and hybrid models have previously tried to forecast COVID-19 confirmed cases for profoundly affected countries. Due to extreme uncertainty and nonstationarity in the time series data, forecasting of COVID-19 confirmed cases has become a very challenging job. For univariate time series forecasting, there are various statistical and machine learning models available in the literature. But, epidemic forecasting has a dubious track record. Its failures became more prominent due to insufficient data input, flaws in modeling assumptions, high sensitivity of estimates, lack of incorporation of epidemiological features, inadequate past evidence on effects of available interventions, lack of transparency, errors, lack of determinacy, and lack of expertise in crucial disciplines. This chapter focuses on assessing different short-term forecasting models that can forecast the daily COVID-19 cases for various countries. In the form of an empirical study on forecasting accuracy, this chapter provides evidence to show that there is no universal method available that can accurately forecast pandemic data. Still, forecasters' predictions are useful for the effective allocation of healthcare resources and will act as an early-warning system for government policymakers.
△ Less
Submitted 10 October, 2020;
originally announced October 2020.