-
Repeating versus Nonrepeating Fast Radio Bursts: A Deep Learning Approach to Morphological Characterization
Authors:
Bikash Kharel,
Emmanuel Fonseca,
Charanjot Brar,
Afrokk Khan,
Lluis Mas-Ribas,
Swarali Shivraj Patil,
Paul Scholz,
Seth Robert Siegel,
David C. Stenning
Abstract:
We present a deep learning approach to classify fast radio bursts (FRBs) based purely on morphology as encoded on recorded dynamic spectrum from CHIME/FRB Catalog 2. We implemented transfer learning with a pretrained ConvNext architecture, exploiting its powerful feature extraction ability. ConvNext was adapted to classify dedispersed dynamic spectra (which we treat as images) of the FRBs into one…
▽ More
We present a deep learning approach to classify fast radio bursts (FRBs) based purely on morphology as encoded on recorded dynamic spectrum from CHIME/FRB Catalog 2. We implemented transfer learning with a pretrained ConvNext architecture, exploiting its powerful feature extraction ability. ConvNext was adapted to classify dedispersed dynamic spectra (which we treat as images) of the FRBs into one of the two sub-classes, i.e., repeater and non-repeater, based on their various temporal and spectral properties and relation between the sub-pulse structures. Additionally, we also used mathematical model representation of the total intensity data to interpret the deep learning model. Upon fine-tuning the pretrained ConvNext on the FRB spectrograms, we were able to achieve high classification metrics while substantially reducing training time and computing power as compared to training a deep learning model from scratch with random weights and biases without any feature extraction ability. Importantly, our results suggest that the morphological differences between CHIME repeating and non-repeating events persist in Catalog 2 and the deep learning model leveraged these differences for classification. The fine-tuned deep learning model can be used for inference, which enables us to predict whether an FRB's morphology resembles that of repeaters or non-repeaters. Such inferences may become increasingly significant when trained on larger data sets that will exist in the near future.
△ Less
Submitted 7 February, 2026; v1 submitted 7 September, 2025;
originally announced September 2025.
-
Recomposer: Event-roll-guided generative audio editing
Authors:
Daniel P. W. Ellis,
Eduardo Fonseca,
Ron J. Weiss,
Kevin Wilson,
Scott Wisdom,
Hakan Erdogan,
John R. Hershey,
Aren Jansen,
R. Channing Moore,
Manoj Plakal
Abstract:
Editing complex real-world sound scenes is difficult because individual sound sources overlap in time. Generative models can fill-in missing or corrupted details based on their strong prior understanding of the data domain. We present a system for editing individual sound events within complex scenes able to delete, insert, and enhance individual sound events based on textual edit descriptions (e.…
▽ More
Editing complex real-world sound scenes is difficult because individual sound sources overlap in time. Generative models can fill-in missing or corrupted details based on their strong prior understanding of the data domain. We present a system for editing individual sound events within complex scenes able to delete, insert, and enhance individual sound events based on textual edit descriptions (e.g., ``enhance Door'') and a graphical representation of the event timing derived from an ``event roll'' transcription. We present an encoder-decoder transformer working on SoundStream representations, trained on synthetic (input, desired output) audio example pairs formed by adding isolated sound events to dense, real-world backgrounds. Evaluation reveals the importance of each part of the edit descriptions -- action, class, timing. Our work demonstrates ``recomposition'' is an important and practical application.
△ Less
Submitted 5 September, 2025;
originally announced September 2025.
-
Dataset balancing can hurt model performance
Authors:
R. Channing Moore,
Daniel P. W. Ellis,
Eduardo Fonseca,
Shawn Hershey,
Aren Jansen,
Manoj Plakal
Abstract:
Machine learning from training data with a skewed distribution of examples per class can lead to models that favor performance on common classes at the expense of performance on rare ones. AudioSet has a very wide range of priors over its 527 sound event classes. Classification performance on AudioSet is usually evaluated by a simple average over per-class metrics, meaning that performance on rare…
▽ More
Machine learning from training data with a skewed distribution of examples per class can lead to models that favor performance on common classes at the expense of performance on rare ones. AudioSet has a very wide range of priors over its 527 sound event classes. Classification performance on AudioSet is usually evaluated by a simple average over per-class metrics, meaning that performance on rare classes is equal in importance to the performance on common ones. Several recent papers have used dataset balancing techniques to improve performance on AudioSet. We find, however, that while balancing improves performance on the public AudioSet evaluation data it simultaneously hurts performance on an unpublished evaluation set collected under the same conditions. By varying the degree of balancing, we show that its benefits are fragile and depend on the evaluation set. We also do not find evidence indicating that balancing improves rare class performance relative to common classes. We therefore caution against blind application of balancing, as well as against paying too much attention to small improvements on a public evaluation set.
△ Less
Submitted 30 June, 2023;
originally announced July 2023.
-
Audiovisual Masked Autoencoders
Authors:
Mariana-Iuliana Georgescu,
Eduardo Fonseca,
Radu Tudor Ionescu,
Mario Lucic,
Cordelia Schmid,
Anurag Arnab
Abstract:
Can we leverage the audiovisual information already present in video to improve self-supervised representation learning? To answer this question, we study various pretraining architectures and objectives within the masked autoencoding framework, motivated by the success of similar methods in natural language and image understanding. We show that we can achieve significant improvements on audiovisu…
▽ More
Can we leverage the audiovisual information already present in video to improve self-supervised representation learning? To answer this question, we study various pretraining architectures and objectives within the masked autoencoding framework, motivated by the success of similar methods in natural language and image understanding. We show that we can achieve significant improvements on audiovisual downstream classification tasks, surpassing the state-of-the-art on VGGSound and AudioSet. Furthermore, we can leverage our audiovisual pretraining scheme for multiple unimodal downstream tasks using a single audiovisual pretrained model. We additionally demonstrate the transferability of our representations, achieving state-of-the-art audiovisual results on Epic Kitchens without pretraining specifically for this dataset.
△ Less
Submitted 4 January, 2024; v1 submitted 9 December, 2022;
originally announced December 2022.
-
Description and analysis of novelties introduced in DCASE Task 4 2022 on the baseline system
Authors:
Francesca Ronchini,
Samuele Cornell,
Romain Serizel,
Nicolas Turpault,
Eduardo Fonseca,
Daniel P. W. Ellis
Abstract:
The aim of the Detection and Classification of Acoustic Scenes and Events Challenge Task 4 is to evaluate systems for the detection of sound events in domestic environments using an heterogeneous dataset. The systems need to be able to correctly detect the sound events present in a recorded audio clip, as well as localize the events in time. This year's task is a follow-up of DCASE 2021 Task 4, wi…
▽ More
The aim of the Detection and Classification of Acoustic Scenes and Events Challenge Task 4 is to evaluate systems for the detection of sound events in domestic environments using an heterogeneous dataset. The systems need to be able to correctly detect the sound events present in a recorded audio clip, as well as localize the events in time. This year's task is a follow-up of DCASE 2021 Task 4, with some important novelties. The goal of this paper is to describe and motivate these new additions, and report an analysis of their impact on the baseline system. We introduced three main novelties: the use of external datasets, including recently released strongly annotated clips from Audioset, the possibility of leveraging pre-trained models, and a new energy consumption metric to raise awareness about the ecological impact of training sound events detectors. The results on the baseline system show that leveraging open-source pretrained on AudioSet improves the results significantly in terms of event classification but not in terms of event segmentation.
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
HEAR: Holistic Evaluation of Audio Representations
Authors:
Joseph Turian,
Jordie Shier,
Humair Raj Khan,
Bhiksha Raj,
Björn W. Schuller,
Christian J. Steinmetz,
Colin Malloy,
George Tzanetakis,
Gissel Velarde,
Kirk McNally,
Max Henry,
Nicolas Pinto,
Camille Noufi,
Christian Clough,
Dorien Herremans,
Eduardo Fonseca,
Jesse Engel,
Justin Salamon,
Philippe Esling,
Pranay Manocha,
Shinji Watanabe,
Zeyu Jin,
Yonatan Bisk
Abstract:
What audio embedding approach generalizes best to a wide range of downstream tasks across a variety of everyday domains without fine-tuning? The aim of the HEAR benchmark is to develop a general-purpose audio representation that provides a strong basis for learning in a wide variety of tasks and scenarios. HEAR evaluates audio representations using a benchmark suite across a variety of domains, in…
▽ More
What audio embedding approach generalizes best to a wide range of downstream tasks across a variety of everyday domains without fine-tuning? The aim of the HEAR benchmark is to develop a general-purpose audio representation that provides a strong basis for learning in a wide variety of tasks and scenarios. HEAR evaluates audio representations using a benchmark suite across a variety of domains, including speech, environmental sound, and music. HEAR was launched as a NeurIPS 2021 shared challenge. In the spirit of shared exchange, each participant submitted an audio embedding model following a common API that is general-purpose, open-source, and freely available to use. Twenty-nine models by thirteen external teams were evaluated on nineteen diverse downstream tasks derived from sixteen datasets. Open evaluation code, submitted models and datasets are key contributions, enabling comprehensive and reproducible evaluation, as well as previously impossible longitudinal studies. It still remains an open question whether one single general-purpose audio representation can perform as holistically as the human ear.
△ Less
Submitted 29 May, 2022; v1 submitted 6 March, 2022;
originally announced March 2022.
-
Predicting Attention Sparsity in Transformers
Authors:
Marcos Treviso,
António Góis,
Patrick Fernandes,
Erick Fonseca,
André F. T. Martins
Abstract:
Transformers' quadratic complexity with respect to the input sequence length has motivated a body of work on efficient sparse approximations to softmax. An alternative path, used by entmax transformers, consists of having built-in exact sparse attention; however this approach still requires quadratic computation. In this paper, we propose Sparsefinder, a simple model trained to identify the sparsi…
▽ More
Transformers' quadratic complexity with respect to the input sequence length has motivated a body of work on efficient sparse approximations to softmax. An alternative path, used by entmax transformers, consists of having built-in exact sparse attention; however this approach still requires quadratic computation. In this paper, we propose Sparsefinder, a simple model trained to identify the sparsity pattern of entmax attention before computing it. We experiment with three variants of our method, based on distances, quantization, and clustering, on two tasks: machine translation (attention in the decoder) and masked language modeling (encoder-only). Our work provides a new angle to study model efficiency by doing extensive analysis of the tradeoff between the sparsity and recall of the predicted attention graph. This allows for detailed comparison between different models along their Pareto curves, important to guide future benchmarks for sparse attention models.
△ Less
Submitted 21 April, 2022; v1 submitted 24 September, 2021;
originally announced September 2021.
-
Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networks
Authors:
Eduardo Fonseca,
Andres Ferraro,
Xavier Serra
Abstract:
Recent studies have put into question the commonly assumed shift invariance property of convolutional networks, showing that small shifts in the input can affect the output predictions substantially. In this paper, we analyze the benefits of addressing lack of shift invariance in CNN-based sound event classification. Specifically, we evaluate two pooling methods to improve shift invariance in CNNs…
▽ More
Recent studies have put into question the commonly assumed shift invariance property of convolutional networks, showing that small shifts in the input can affect the output predictions substantially. In this paper, we analyze the benefits of addressing lack of shift invariance in CNN-based sound event classification. Specifically, we evaluate two pooling methods to improve shift invariance in CNNs, based on low-pass filtering and adaptive sampling of incoming feature maps. These methods are implemented via small architectural modifications inserted into the pooling layers of CNNs. We evaluate the effect of these architectural changes on the FSD50K dataset using models of different capacity and in presence of strong regularization. We show that these modifications consistently improve sound event classification in all cases considered. We also demonstrate empirically that the proposed pooling methods increase shift invariance in the network, making it more robust against time/frequency shifts in input spectrograms. This is achieved by adding a negligible amount of trainable parameters, which makes these methods an appealing alternative to conventional pooling layers. The outcome is a new state-of-the-art mAP of 0.541 on the FSD50K classification benchmark.
△ Less
Submitted 22 July, 2021; v1 submitted 1 July, 2021;
originally announced July 2021.
-
The Benefit Of Temporally-Strong Labels In Audio Event Classification
Authors:
Shawn Hershey,
Daniel P W Ellis,
Eduardo Fonseca,
Aren Jansen,
Caroline Liu,
R Channing Moore,
Manoj Plakal
Abstract:
To reveal the importance of temporal precision in ground truth audio event labels, we collected precise (~0.1 sec resolution) "strong" labels for a portion of the AudioSet dataset. We devised a temporally strong evaluation set (including explicit negatives of varying difficulty) and a small strong-labeled training subset of 67k clips (compared to the original dataset's 1.8M clips labeled at 10 sec…
▽ More
To reveal the importance of temporal precision in ground truth audio event labels, we collected precise (~0.1 sec resolution) "strong" labels for a portion of the AudioSet dataset. We devised a temporally strong evaluation set (including explicit negatives of varying difficulty) and a small strong-labeled training subset of 67k clips (compared to the original dataset's 1.8M clips labeled at 10 sec resolution). We show that fine-tuning with a mix of weak and strongly labeled data can substantially improve classifier performance, even when evaluated using only the original weak labels. For a ResNet50 architecture, d' on the strong evaluation data including explicit negatives improves from 1.13 to 1.41. The new labels are available as an update to AudioSet.
△ Less
Submitted 14 May, 2021;
originally announced May 2021.
-
Self-Supervised Learning from Automatically Separated Sound Scenes
Authors:
Eduardo Fonseca,
Aren Jansen,
Daniel P. W. Ellis,
Scott Wisdom,
Marco Tagliasacchi,
John R. Hershey,
Manoj Plakal,
Shawn Hershey,
R. Channing Moore,
Xavier Serra
Abstract:
Real-world sound scenes consist of time-varying collections of sound sources, each generating characteristic sound events that are mixed together in audio recordings. The association of these constituent sound events with their mixture and each other is semantically constrained: the sound scene contains the union of source classes and not all classes naturally co-occur. With this motivation, this…
▽ More
Real-world sound scenes consist of time-varying collections of sound sources, each generating characteristic sound events that are mixed together in audio recordings. The association of these constituent sound events with their mixture and each other is semantically constrained: the sound scene contains the union of source classes and not all classes naturally co-occur. With this motivation, this paper explores the use of unsupervised automatic sound separation to decompose unlabeled sound scenes into multiple semantically-linked views for use in self-supervised contrastive learning. We find that learning to associate input mixtures with their automatically separated outputs yields stronger representations than past approaches that use the mixtures alone. Further, we discover that optimal source separation is not required for successful contrastive learning by demonstrating that a range of separation system convergence states all lead to useful and often complementary example transformations. Our best system incorporates these unsupervised separation models into a single augmentation front-end and jointly optimizes similarity maximization and coincidence prediction objectives across the views. The result is an unsupervised audio representation that rivals state-of-the-art alternatives on the established shallow AudioSet classification benchmark.
△ Less
Submitted 14 September, 2021; v1 submitted 5 May, 2021;
originally announced May 2021.
-
Mobility for Cellular-Connected UAVs: challenges for the network provider
Authors:
Erika Fonseca,
Boris Galkin,
Marvin Kelly,
Luiz A. DaSilva,
Ivana Dusparic
Abstract:
Unmanned Aerial Vehicle (UAV) technology is becoming more prevalent and more diverse in its application. 5G and beyond networks must enable UAV connectivity. This will require the network operator to consider this new type of user in the planning and operation of the network. This work presents the challenges an operator will encounter and should consider in the future as UAVs become users of the…
▽ More
Unmanned Aerial Vehicle (UAV) technology is becoming more prevalent and more diverse in its application. 5G and beyond networks must enable UAV connectivity. This will require the network operator to consider this new type of user in the planning and operation of the network. This work presents the challenges an operator will encounter and should consider in the future as UAVs become users of the network. We analyse the 3GPP specifications, the existing research literature, and a publicly available UAV connectivity dataset, to describe the challenges. We classify these challenges into network planning and network optimisation categories. We discuss the challenge of planning network coverage when considering coverage for flying users and the PCI collision and confusion issues that can be aggravated by these users. In discussing network optimisation challenges, we introduce Automatic Neighbouring Relation (ANR) and handover challenges, specifically the number of neighbours in the Neighbour Relation Table (NRT), and their potential deletion and block-listing, the frequent number of handovers and the possibility that the UAV disconnects because of handover issues. We discuss possible approaches to address the presented challenges and use a real-world dataset to support our findings about these challenges and their importance.
△ Less
Submitted 25 February, 2021;
originally announced February 2021.
-
Unsupervised Contrastive Learning of Sound Event Representations
Authors:
Eduardo Fonseca,
Diego Ortego,
Kevin McGuinness,
Noel E. O'Connor,
Xavier Serra
Abstract:
Self-supervised representation learning can mitigate the limitations in recognition tasks with few manually labeled data but abundant unlabeled data---a common scenario in sound event research. In this work, we explore unsupervised contrastive learning as a way to learn sound event representations. To this end, we propose to use the pretext task of contrasting differently augmented views of sound…
▽ More
Self-supervised representation learning can mitigate the limitations in recognition tasks with few manually labeled data but abundant unlabeled data---a common scenario in sound event research. In this work, we explore unsupervised contrastive learning as a way to learn sound event representations. To this end, we propose to use the pretext task of contrasting differently augmented views of sound events. The views are computed primarily via mixing of training examples with unrelated backgrounds, followed by other data augmentations. We analyze the main components of our method via ablation experiments. We evaluate the learned representations using linear evaluation, and in two in-domain downstream sound event classification tasks, namely, using limited manually labeled data, and using noisy labeled data. Our results suggest that unsupervised contrastive pre-training can mitigate the impact of data scarcity and increase robustness against noisy labels, outperforming supervised baselines.
△ Less
Submitted 15 November, 2020;
originally announced November 2020.
-
Experimental Evaluation of a UAV User QoS from a Two-Tier 3.6GHz Spectrum Network
Authors:
Boris Galkin,
Erika Fonseca,
Gavin Lee,
Conor Duff,
Marvin Kelly,
Edward Emmanuel,
Ivana Dusparic
Abstract:
Unmanned Aerial Vehicle (UAV) technology is becoming increasingly used in a variety of applications such as video surveillance and deliveries. To enable safe and efficient use of UAVs, the devices will need to be connected into cellular networks. Existing research on UAV cellular connectivity shows that UAVs encounter significant issues with existing networks, such as strong interference and anten…
▽ More
Unmanned Aerial Vehicle (UAV) technology is becoming increasingly used in a variety of applications such as video surveillance and deliveries. To enable safe and efficient use of UAVs, the devices will need to be connected into cellular networks. Existing research on UAV cellular connectivity shows that UAVs encounter significant issues with existing networks, such as strong interference and antenna misalignment. In this work, we perform a novel measurement campaign of the performance of a UAV user when it connects to an experimental two-tier cellular network in two different areas of Dublin city's Smart Docklands, which includes massive MIMO macrocells and wirelessly-backhauled small cells. We measure Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ), Signal to Interference and Noise Ratio (SINR), the downlink throughput, and the small cell handover rate. Our results show that increasing the UAV height reduces the performance in both tiers, due to issues such as antenna misalignment. The small cell tier, however, can maintain relatively stable performance across the entire range of UAV heights, suggesting that UAV users can successfully connect to small cells during their flight. Furthermore, we demonstrate that while the UAV handover rate significantly fluctuates at different heights, the overall observed handover rates are very low. Our results highlight the potential for small cells in urban areas to provide connectivity to UAVs.
△ Less
Submitted 9 April, 2021; v1 submitted 6 November, 2020;
originally announced November 2020.
-
What's All the FUSS About Free Universal Sound Separation Data?
Authors:
Scott Wisdom,
Hakan Erdogan,
Daniel Ellis,
Romain Serizel,
Nicolas Turpault,
Eduardo Fonseca,
Justin Salamon,
Prem Seetharaman,
John Hershey
Abstract:
We introduce the Free Universal Sound Separation (FUSS) dataset, a new corpus for experiments in separating mixtures of an unknown number of sounds from an open domain of sound types. The dataset consists of 23 hours of single-source audio data drawn from 357 classes, which are used to create mixtures of one to four sources. To simulate reverberation, an acoustic room simulator is used to generate…
▽ More
We introduce the Free Universal Sound Separation (FUSS) dataset, a new corpus for experiments in separating mixtures of an unknown number of sounds from an open domain of sound types. The dataset consists of 23 hours of single-source audio data drawn from 357 classes, which are used to create mixtures of one to four sources. To simulate reverberation, an acoustic room simulator is used to generate impulse responses of box shaped rooms with frequency-dependent reflective walls. Additional open-source data augmentation tools are also provided to produce new mixtures with different combinations of sources and room simulations. Finally, we introduce an open-source baseline separation model, based on an improved time-domain convolutional network (TDCN++), that can separate a variable number of sources in a mixture. This model achieves 9.8 dB of scale-invariant signal-to-noise ratio improvement (SI-SNRi) on mixtures with two to four sources, while reconstructing single-source inputs with 35.5 dB absolute SI-SNR. We hope this dataset will lower the barrier to new research and allow for fast iteration and application of novel techniques from other machine learning domains to the sound separation challenge.
△ Less
Submitted 2 November, 2020;
originally announced November 2020.
-
Sound Event Detection and Separation: a Benchmark on Desed Synthetic Soundscapes
Authors:
Nicolas Turpault,
Romain Serizel,
Scott Wisdom,
Hakan Erdogan,
John Hershey,
Eduardo Fonseca,
Prem Seetharaman,
Justin Salamon
Abstract:
We propose a benchmark of state-of-the-art sound event detection systems (SED). We designed synthetic evaluation sets to focus on specific sound event detection challenges. We analyze the performance of the submissions to DCASE 2021 task 4 depending on time related modifications (time position of an event and length of clips) and we study the impact of non-target sound events and reverberation. We…
▽ More
We propose a benchmark of state-of-the-art sound event detection systems (SED). We designed synthetic evaluation sets to focus on specific sound event detection challenges. We analyze the performance of the submissions to DCASE 2021 task 4 depending on time related modifications (time position of an event and length of clips) and we study the impact of non-target sound events and reverberation. We show that the localization in time of sound events is still a problem for SED systems. We also show that reverberation and non-target sound events are severely degrading the performance of the SED systems. In the latter case, sound separation seems like a promising solution.
△ Less
Submitted 2 November, 2020;
originally announced November 2020.
-
MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset
Authors:
Marina Fomicheva,
Shuo Sun,
Erick Fonseca,
Chrysoula Zerva,
Frédéric Blain,
Vishrav Chaudhary,
Francisco Guzmán,
Nina Lopatina,
Lucia Specia,
André F. T. Martins
Abstract:
We present MLQE-PE, a new dataset for Machine Translation (MT) Quality Estimation (QE) and Automatic Post-Editing (APE). The dataset contains eleven language pairs, with human labels for up to 10,000 translations per language pair in the following formats: sentence-level direct assessments and post-editing effort, and word-level good/bad labels. It also contains the post-edited sentences, as well…
▽ More
We present MLQE-PE, a new dataset for Machine Translation (MT) Quality Estimation (QE) and Automatic Post-Editing (APE). The dataset contains eleven language pairs, with human labels for up to 10,000 translations per language pair in the following formats: sentence-level direct assessments and post-editing effort, and word-level good/bad labels. It also contains the post-edited sentences, as well as titles of the articles where the sentences were extracted from, and the neural MT models used to translate the text.
△ Less
Submitted 11 October, 2021; v1 submitted 9 October, 2020;
originally announced October 2020.
-
REQIBA: Regression and Deep Q-Learning for Intelligent UAV Cellular User to Base Station Association
Authors:
Boris Galkin,
Erika Fonseca,
Ramy Amer,
Luiz A. DaSilva,
Ivana Dusparic
Abstract:
Unmanned Aerial Vehicles (UAVs) are emerging as important users of next-generation cellular networks. By operating in the sky, UAV users experience very different radio conditions than terrestrial users, due to factors such as strong Line-of-Sight (LoS) channels (and interference) and Base Station (BS) antenna misalignment. As a consequence, the UAVs may experience significant degradation to their…
▽ More
Unmanned Aerial Vehicles (UAVs) are emerging as important users of next-generation cellular networks. By operating in the sky, UAV users experience very different radio conditions than terrestrial users, due to factors such as strong Line-of-Sight (LoS) channels (and interference) and Base Station (BS) antenna misalignment. As a consequence, the UAVs may experience significant degradation to their received quality of service, particularly when they are moving and are subject to frequent handovers. The solution is to allow the UAV to be aware of its surrounding environment, and intelligently connect into the cellular network taking advantage of this awareness. In this paper we present REgression and deep Q-learning for Intelligent UAV cellular user to Base station Association (REQIBA), a solution that allows a UAV flying over an urban area to intelligently connect to underlying BSs, using information about the received signal powers, the BS locations, and the surrounding building topology. We demonstrate how REQIBA can as much as double the total UAV throughput, when compared to heuristic association schemes similar to those commonly used by terrestrial users. We also evaluate how environmental factors such as UAV height, building density, and throughput loss due to handovers impact the performance of our solution.
△ Less
Submitted 3 November, 2021; v1 submitted 2 October, 2020;
originally announced October 2020.
-
FSD50K: An Open Dataset of Human-Labeled Sound Events
Authors:
Eduardo Fonseca,
Xavier Favory,
Jordi Pons,
Frederic Font,
Xavier Serra
Abstract:
Most existing datasets for sound event recognition (SER) are relatively small and/or domain-specific, with the exception of AudioSet, based on over 2M tracks from YouTube videos and encompassing over 500 sound classes. However, AudioSet is not an open dataset as its official release consists of pre-computed audio features. Downloading the original audio tracks can be problematic due to YouTube vid…
▽ More
Most existing datasets for sound event recognition (SER) are relatively small and/or domain-specific, with the exception of AudioSet, based on over 2M tracks from YouTube videos and encompassing over 500 sound classes. However, AudioSet is not an open dataset as its official release consists of pre-computed audio features. Downloading the original audio tracks can be problematic due to YouTube videos gradually disappearing and usage rights issues. To provide an alternative benchmark dataset and thus foster SER research, we introduce FSD50K, an open dataset containing over 51k audio clips totalling over 100h of audio manually labeled using 200 classes drawn from the AudioSet Ontology. The audio clips are licensed under Creative Commons licenses, making the dataset freely distributable (including waveforms). We provide a detailed description of the FSD50K creation process, tailored to the particularities of Freesound data, including challenges encountered and solutions adopted. We include a comprehensive dataset characterization along with discussion of limitations and key factors to allow its audio-informed usage. Finally, we conduct sound event classification experiments to provide baseline systems as well as insight on the main factors to consider when splitting Freesound audio data for SER. Our goal is to develop a dataset to be widely adopted by the community as a new open benchmark for SER research.
△ Less
Submitted 23 April, 2022; v1 submitted 1 October, 2020;
originally announced October 2020.
-
Adaptive Height Optimisation for Cellular-Connected UAVs using Reinforcement Learning
Authors:
Erika Fonseca,
Boris Galkin,
Ramy Amer,
Luiz A. DaSilva,
Ivana Dusparic
Abstract:
Providing reliable connectivity to cellular-connected UAV can be very challenging; their performance highly depends on the nature of the surrounding environment, such as density and heights of the ground BSs. On the other hand, tall buildings might block undesired interference signals from ground BSs, thereby improving the connectivity between the UAVs and their serving BSs. To address the connect…
▽ More
Providing reliable connectivity to cellular-connected UAV can be very challenging; their performance highly depends on the nature of the surrounding environment, such as density and heights of the ground BSs. On the other hand, tall buildings might block undesired interference signals from ground BSs, thereby improving the connectivity between the UAVs and their serving BSs. To address the connectivity of UAVs in such environments, this paper proposes a RL algorithm to dynamically optimise the height of a UAV as it moves through the environment, with the goal of increasing the throughput or spectrum efficiency that it experiences. The proposed solution is evaluated in two settings: using a series of generated environments where we vary the number of BS and building densities, and in a scenario using real-world data obtained from an experiment in Dublin, Ireland. Results show that our proposed RL-based solution improves UAVs QoS by 6% to 41%, depending on the scenario. We also conclude that, when flying at heights higher than the buildings, building density variation has no impact on UAV QoS. On the other hand, BS density can negatively impact UAV QoS, with higher numbers of BSs generating more interference and deteriorating UAV performance.
△ Less
Submitted 13 April, 2022; v1 submitted 27 July, 2020;
originally announced July 2020.
-
Radio Access Technology Characterisation Through Object Detection
Authors:
Erika Fonseca,
Joao F. Santos,
Francisco Paisana,
Luiz A. DaSilva
Abstract:
\ac{RAT} classification and monitoring are essential for efficient coexistence of different communication systems in shared spectrum. Shared spectrum, including operation in license-exempt bands, is envisioned in the \ac{5G} standards (e.g., 3GPP Rel. 16). In this paper, we propose a \ac{ML} approach to characterise the spectrum utilisation and facilitate the dynamic access to it. Recent advances…
▽ More
\ac{RAT} classification and monitoring are essential for efficient coexistence of different communication systems in shared spectrum. Shared spectrum, including operation in license-exempt bands, is envisioned in the \ac{5G} standards (e.g., 3GPP Rel. 16). In this paper, we propose a \ac{ML} approach to characterise the spectrum utilisation and facilitate the dynamic access to it. Recent advances in \acp{CNN} enable us to perform waveform classification by processing spectrograms as images. In contrast to other \ac{ML} methods that can only provide the class of the monitored \acp{RAT}, the solution we propose can recognise not only different \acp{RAT} in shared spectrum, but also identify critical parameters such as inter-frame duration, frame duration, centre frequency, and signal bandwidth by using object detection and a feature extraction module to extract features from spectrograms. We have implemented and evaluated our solution using a dataset of commercial transmissions, as well as in a \ac{SDR} testbed environment. The scenario evaluated was the coexistence of WiFi and LTE transmissions in shared spectrum. Our results show that our approach has an accuracy of 96\% in the classification of \acp{RAT} from a dataset that captures transmissions of regular user communications. It also shows that the extracted features can be precise within a margin of 2\%, %of the size of the image, and is capable of detect above 94\% of objects under a broad range of transmission power levels and interference conditions.
△ Less
Submitted 27 July, 2020;
originally announced July 2020.
-
Improving Sound Event Detection In Domestic Environments Using Sound Separation
Authors:
Nicolas Turpault,
Scott Wisdom,
Hakan Erdogan,
John Hershey,
Romain Serizel,
Eduardo Fonseca,
Prem Seetharaman,
Justin Salamon
Abstract:
Performing sound event detection on real-world recordings often implies dealing with overlapping target sound events and non-target sounds, also referred to as interference or noise. Until now these problems were mainly tackled at the classifier level. We propose to use sound separation as a pre-processing for sound event detection. In this paper we start from a sound separation model trained on t…
▽ More
Performing sound event detection on real-world recordings often implies dealing with overlapping target sound events and non-target sounds, also referred to as interference or noise. Until now these problems were mainly tackled at the classifier level. We propose to use sound separation as a pre-processing for sound event detection. In this paper we start from a sound separation model trained on the Free Universal Sound Separation dataset and the DCASE 2020 task 4 sound event detection baseline. We explore different methods to combine separated sound sources and the original mixture within the sound event detection. Furthermore, we investigate the impact of adapting the sound separation model to the sound event detection data on both the sound separation and the sound event detection.
△ Less
Submitted 8 July, 2020;
originally announced July 2020.
-
Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking
Authors:
Eduardo Fonseca,
Shawn Hershey,
Manoj Plakal,
Daniel P. W. Ellis,
Aren Jansen,
R. Channing Moore,
Xavier Serra
Abstract:
The study of label noise in sound event recognition has recently gained attention with the advent of larger and noisier datasets. This work addresses the problem of missing labels, one of the big weaknesses of large audio datasets, and one of the most conspicuous issues for AudioSet. We propose a simple and model-agnostic method based on a teacher-student framework with loss masking to first ident…
▽ More
The study of label noise in sound event recognition has recently gained attention with the advent of larger and noisier datasets. This work addresses the problem of missing labels, one of the big weaknesses of large audio datasets, and one of the most conspicuous issues for AudioSet. We propose a simple and model-agnostic method based on a teacher-student framework with loss masking to first identify the most critical missing label candidates, and then ignore their contribution during the learning process. We find that a simple optimisation of the training label set improves recognition performance without additional computation. We discover that most of the improvement comes from ignoring a critical tiny portion of the missing labels. We also show that the damage done by missing labels is larger as the training set gets smaller, yet it can still be observed even when training with massive amounts of audio. We believe these insights can generalize to other large-scale datasets.
△ Less
Submitted 25 July, 2020; v1 submitted 2 May, 2020;
originally announced May 2020.
-
Intelligent Base Station Association for UAV Cellular Users: A Supervised Learning Approach
Authors:
Boris Galkin,
Ramy Amer,
Erika Fonseca,
Luiz A. DaSilva
Abstract:
Fifth Generation (5G) cellular networks are expected to provide cellular connectivity for vehicular users, including Unmanned Aerial Vehicles (UAVs). When flying in the air, these users experience strong, unobstructed channel conditions to a large number of Base Stations (BSs) on the ground. This creates very strong interference conditions for the UAV users, while at the same time offering them a…
▽ More
Fifth Generation (5G) cellular networks are expected to provide cellular connectivity for vehicular users, including Unmanned Aerial Vehicles (UAVs). When flying in the air, these users experience strong, unobstructed channel conditions to a large number of Base Stations (BSs) on the ground. This creates very strong interference conditions for the UAV users, while at the same time offering them a large number of BSs to potentially associate with for cellular service. Therefore, to maximise the performance of the UAV-BS wireless link, the UAV user needs to be able to choose which BSs to connect to, based on the observed environmental conditions. This paper proposes a supervised learning-based association scheme, using which a UAV can intelligently associate with the most appropriate BS. We train a Neural Network (NN) to identify the most suitable BS from several candidate BSs, based on the received signal powers from the BSs, known distances to the BSs, as well as the known locations of potential interferers. We then compare the performance of the NN-based association scheme against strongest-signal and closest-neighbour association schemes, and demonstrate that the NN scheme significantly outperforms the simple heuristic schemes.
△ Less
Submitted 28 August, 2020; v1 submitted 2 March, 2020;
originally announced March 2020.
-
Model-agnostic Approaches to Handling Noisy Labels When Training Sound Event Classifiers
Authors:
Eduardo Fonseca,
Frederic Font,
Xavier Serra
Abstract:
Label noise is emerging as a pressing issue in sound event classification. This arises as we move towards larger datasets that are difficult to annotate manually, but it is even more severe if datasets are collected automatically from online repositories, where labels are inferred through automated heuristics applied to the audio content or metadata. While learning from noisy labels has been an ac…
▽ More
Label noise is emerging as a pressing issue in sound event classification. This arises as we move towards larger datasets that are difficult to annotate manually, but it is even more severe if datasets are collected automatically from online repositories, where labels are inferred through automated heuristics applied to the audio content or metadata. While learning from noisy labels has been an active area of research in computer vision, it has received little attention in sound event classification. Most recent computer vision approaches against label noise are relatively complex, requiring complex networks or extra data resources. In this work, we evaluate simple and efficient model-agnostic approaches to handling noisy labels when training sound event classifiers, namely label smoothing regularization, mixup and noise-robust loss functions. The main advantage of these methods is that they can be easily incorporated to existing deep learning pipelines without need for network modifications or extra resources. We report results from experiments conducted with the FSDnoisy18k dataset. We show that these simple methods can be effective in mitigating the effect of label noise, providing up to 2.5\% of accuracy boost when incorporated to two different CNNs, while requiring minimal intervention and computational overhead.
△ Less
Submitted 26 October, 2019;
originally announced October 2019.
-
A hybrid parametric-deep learning approach for sound event localization and detection
Authors:
Andres Perez-Lopez,
Eduardo Fonseca,
Xavier Serra
Abstract:
This work describes and discusses an algorithm submitted to the Sound Event Localization and Detection Task of DCASE2019 Challenge. The proposed methodology relies on parametric spatial audio analysis for source localization and detection, combined with a deep learning-based monophonic event classifier. The evaluation of the proposed algorithm yields overall results comparable to the baseline syst…
▽ More
This work describes and discusses an algorithm submitted to the Sound Event Localization and Detection Task of DCASE2019 Challenge. The proposed methodology relies on parametric spatial audio analysis for source localization and detection, combined with a deep learning-based monophonic event classifier. The evaluation of the proposed algorithm yields overall results comparable to the baseline system. The main highlight is a reduction of the localization error on the evaluation dataset by a factor of 2.6, compared with the baseline performance.
△ Less
Submitted 27 August, 2019;
originally announced August 2019.
-
Agro 4.0: A Green Information System for Sustainable Agroecosystem Management
Authors:
Eugênio Pacceli Reis da Fonseca,
Evandro Caldeira,
Heitor Soares Ramos Filho,
Leonardo Barbosa e Oliveira,
Adriano César Machado Pereira,
Pierre Santos Vilela
Abstract:
Agriculture is one of the most critical activities developed today by humankind and is in constant technical evolution to supply food and other essential products to everlasting and increasing demand. New machines, seeds, and fertilizers were developed to increase the productivity of cultivated areas. It is estimated that by 2050 we will have a population of 9 billion people and the production of…
▽ More
Agriculture is one of the most critical activities developed today by humankind and is in constant technical evolution to supply food and other essential products to everlasting and increasing demand. New machines, seeds, and fertilizers were developed to increase the productivity of cultivated areas. It is estimated that by 2050 we will have a population of 9 billion people and the production of food to meet this demand must occur sustainably. To achieve this goal, it is paramount the adoption of sustainable management techniques for agroecosystems. However, this is a complex task due to a large number of variables involved. One of the solutions for the handling and treatment of such diverse data is the use of Green IS. In this work, we adopt a methodology called Indicators of Sustainability in Agroecosystems (Indicadores de Sustentabilidade em Agroecossistemas -- ISA), implement an information system based on it and apply Data Science techniques over the gathered data - from 100 real rural properties - to compute which are the most relevant ISA Indicators for the final ISA Sustainability Index Score. As a result, we have developed a set of tools for data collection, processing, visualization, and analysis of the sustainability of a rural property or region, following the ISA methodology. We also have that with only 7 of the 21 Indicators present in ISA we can identify the level of sustainability in more than 90% of cases, allowing for a new discussion about shrinking the amount of data needed for the computation of ISA, or remodelling the final computation of the Sustainability Index so other Indicators can be more expressive. Users of the solutions developed in this work can identify best practices for sustainability in participating agroecosystems.
△ Less
Submitted 11 July, 2019;
originally announced July 2019.
-
Audio tagging with noisy labels and minimal supervision
Authors:
Eduardo Fonseca,
Manoj Plakal,
Frederic Font,
Daniel P. W. Ellis,
Xavier Serra
Abstract:
This paper introduces Task 2 of the DCASE2019 Challenge, titled "Audio tagging with noisy labels and minimal supervision". This task was hosted on the Kaggle platform as "Freesound Audio Tagging 2019". The task evaluates systems for multi-label audio tagging using a large set of noisy-labeled data, and a much smaller set of manually-labeled data, under a large vocabulary setting of 80 everyday sou…
▽ More
This paper introduces Task 2 of the DCASE2019 Challenge, titled "Audio tagging with noisy labels and minimal supervision". This task was hosted on the Kaggle platform as "Freesound Audio Tagging 2019". The task evaluates systems for multi-label audio tagging using a large set of noisy-labeled data, and a much smaller set of manually-labeled data, under a large vocabulary setting of 80 everyday sound classes. In addition, the proposed dataset poses an acoustic mismatch problem between the noisy train set and the test set due to the fact that they come from different web audio sources. This can correspond to a realistic scenario given by the difficulty in gathering large amounts of manually labeled data. We present the task setup, the FSDKaggle2019 dataset prepared for this scientific evaluation, and a baseline system consisting of a convolutional neural network. All these resources are freely available.
△ Less
Submitted 19 January, 2020; v1 submitted 7 June, 2019;
originally announced June 2019.
-
Learning Sound Event Classifiers from Web Audio with Noisy Labels
Authors:
Eduardo Fonseca,
Manoj Plakal,
Daniel P. W. Ellis,
Frederic Font,
Xavier Favory,
Xavier Serra
Abstract:
As sound event classification moves towards larger datasets, issues of label noise become inevitable. Web sites can supply large volumes of user-contributed audio and metadata, but inferring labels from this metadata introduces errors due to unreliable inputs, and limitations in the mapping. There is, however, little research into the impact of these errors. To foster the investigation of label no…
▽ More
As sound event classification moves towards larger datasets, issues of label noise become inevitable. Web sites can supply large volumes of user-contributed audio and metadata, but inferring labels from this metadata introduces errors due to unreliable inputs, and limitations in the mapping. There is, however, little research into the impact of these errors. To foster the investigation of label noise in sound event classification we present FSDnoisy18k, a dataset containing 42.5 hours of audio across 20 sound classes, including a small amount of manually-labeled data and a larger quantity of real-world noisy data. We characterize the label noise empirically, and provide a CNN baseline system. Experiments suggest that training with large amounts of noisy data can outperform training with smaller amounts of carefully-labeled data. We also show that noise-robust loss functions can be effective in improving performance in presence of corrupted labels.
△ Less
Submitted 7 March, 2019; v1 submitted 4 January, 2019;
originally announced January 2019.
-
Facilitating the Manual Annotation of Sounds When Using Large Taxonomies
Authors:
Xavier Favory,
Eduardo Fonseca,
Frederic Font,
Xavier Serra
Abstract:
Properly annotated multimedia content is crucial for supporting advances in many Information Retrieval applications. It enables, for instance, the development of automatic tools for the annotation of large and diverse multimedia collections. In the context of everyday sounds and online collections, the content to describe is very diverse and involves many different types of concepts, often organis…
▽ More
Properly annotated multimedia content is crucial for supporting advances in many Information Retrieval applications. It enables, for instance, the development of automatic tools for the annotation of large and diverse multimedia collections. In the context of everyday sounds and online collections, the content to describe is very diverse and involves many different types of concepts, often organised in large hierarchical structures called taxonomies. This makes the task of manually annotating content arduous. In this paper, we present our user-centered development of two tools for the manual annotation of audio content from a wide range of types. We conducted a preliminary evaluation of functional prototypes involving real users. The goal is to evaluate them in a real context, engage in discussions with users, and inspire new ideas. A qualitative analysis was carried out including usability questionnaires and semi-structured interviews. This revealed interesting aspects to consider when developing tools for the manual annotation of audio content with labels drawn from large hierarchical taxonomies.
△ Less
Submitted 21 November, 2018;
originally announced November 2018.
-
General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline
Authors:
Eduardo Fonseca,
Manoj Plakal,
Frederic Font,
Daniel P. W. Ellis,
Xavier Favory,
Jordi Pons,
Xavier Serra
Abstract:
This paper describes Task 2 of the DCASE 2018 Challenge, titled "General-purpose audio tagging of Freesound content with AudioSet labels". This task was hosted on the Kaggle platform as "Freesound General-Purpose Audio Tagging Challenge". The goal of the task is to build an audio tagging system that can recognize the category of an audio clip from a subset of 41 diverse categories drawn from the A…
▽ More
This paper describes Task 2 of the DCASE 2018 Challenge, titled "General-purpose audio tagging of Freesound content with AudioSet labels". This task was hosted on the Kaggle platform as "Freesound General-Purpose Audio Tagging Challenge". The goal of the task is to build an audio tagging system that can recognize the category of an audio clip from a subset of 41 diverse categories drawn from the AudioSet Ontology. We present the task, the dataset prepared for the competition, and a baseline system.
△ Less
Submitted 6 October, 2018; v1 submitted 25 July, 2018;
originally announced July 2018.
-
A Simple Fusion of Deep and Shallow Learning for Acoustic Scene Classification
Authors:
Eduardo Fonseca,
Rong Gong,
Xavier Serra
Abstract:
In the past, Acoustic Scene Classification systems have been based on hand crafting audio features that are input to a classifier. Nowadays, the common trend is to adopt data driven techniques, e.g., deep learning, where audio representations are learned from data. In this paper, we propose a system that consists of a simple fusion of two methods of the aforementioned types: a deep learning approa…
▽ More
In the past, Acoustic Scene Classification systems have been based on hand crafting audio features that are input to a classifier. Nowadays, the common trend is to adopt data driven techniques, e.g., deep learning, where audio representations are learned from data. In this paper, we propose a system that consists of a simple fusion of two methods of the aforementioned types: a deep learning approach where log-scaled mel-spectrograms are input to a convolutional neural network, and a feature engineering approach, where a collection of hand-crafted features is input to a gradient boosting machine. We first show that both methods provide complementary information to some extent. Then, we use a simple late fusion strategy to combine both methods. We report classification accuracy of each method individually and the combined system on the TUT Acoustic Scenes 2017 dataset. The proposed fused system outperforms each of the individual methods and attains a classification accuracy of 72.8% on the evaluation set, improving the baseline system by 11.8%.
△ Less
Submitted 27 June, 2018; v1 submitted 19 June, 2018;
originally announced June 2018.
-
MilkQA: a Dataset of Consumer Questions for the Task of Answer Selection
Authors:
Marcelo Criscuolo,
Erick Rocha Fonseca,
Sandra Maria Aluísio,
Ana Carolina Sperança-Criscuolo
Abstract:
We introduce MilkQA, a question answering dataset from the dairy domain dedicated to the study of consumer questions. The dataset contains 2,657 pairs of questions and answers, written in the Portuguese language and originally collected by the Brazilian Agricultural Research Corporation (Embrapa). All questions were motivated by real situations and written by thousands of authors with very differe…
▽ More
We introduce MilkQA, a question answering dataset from the dairy domain dedicated to the study of consumer questions. The dataset contains 2,657 pairs of questions and answers, written in the Portuguese language and originally collected by the Brazilian Agricultural Research Corporation (Embrapa). All questions were motivated by real situations and written by thousands of authors with very different backgrounds and levels of literacy, while answers were elaborated by specialists from Embrapa's customer service. Our dataset was filtered and anonymized by three human annotators. Consumer questions are a challenging kind of question that is usually employed as a form of seeking information. Although several question answering datasets are available, most of such resources are not suitable for research on answer selection models for consumer questions. We aim to fill this gap by making MilkQA publicly available. We study the behavior of four answer selection models on MilkQA: two baseline models and two convolutional neural network archictetures. Our results show that MilkQA poses real challenges to computational models, particularly due to linguistic characteristics of its questions and to their unusually longer lengths. Only one of the experimented models gives reasonable results, at the cost of high computational requirements.
△ Less
Submitted 10 January, 2018;
originally announced January 2018.
-
On the Accuracy of Formal Verification of Selective Defenses for TDoS Attacks
Authors:
Marcilio O. O. Lemos,
Yuri Gil Dantas,
Iguatemi E. Fonseca,
Vivek Nigam
Abstract:
Telephony Denial of Service (TDoS) attacks target telephony services, such as Voice over IP (VoIP), not allowing legitimate users to make calls. There are few defenses that attempt to mitigate TDoS attacks, most of them using IP filtering, with limited applicability. In our previous work, we proposed to use selective strategies for mitigating HTTP Application-Layer DDoS Attacks demonstrating their…
▽ More
Telephony Denial of Service (TDoS) attacks target telephony services, such as Voice over IP (VoIP), not allowing legitimate users to make calls. There are few defenses that attempt to mitigate TDoS attacks, most of them using IP filtering, with limited applicability. In our previous work, we proposed to use selective strategies for mitigating HTTP Application-Layer DDoS Attacks demonstrating their effectiveness in mitigating different types of attacks. Developing such types of defenses is challenging as there are many design options, eg, which dropping functions and selection algorithms to use. Our first contribution is to demonstrate both experimentally and by using formal verification that selective strategies are suitable for mitigating TDoS attacks. We used our formal model to help decide which selective strategies to use with much less effort than carrying out experiments. Our second contribution is a detailed comparison of the results obtained from our formal models and the results obtained by carrying out experiments. We demonstrate that formal methods is a powerful tool for specifying defenses for mitigating Distributed Denial of Service attacks allowing to increase our confidence on the proposed defense before actual implementation.
△ Less
Submitted 13 September, 2017;
originally announced September 2017.
-
Portuguese Word Embeddings: Evaluating on Word Analogies and Natural Language Tasks
Authors:
Nathan Hartmann,
Erick Fonseca,
Christopher Shulby,
Marcos Treviso,
Jessica Rodrigues,
Sandra Aluisio
Abstract:
Word embeddings have been found to provide meaningful representations for words in an efficient way; therefore, they have become common in Natural Language Processing sys- tems. In this paper, we evaluated different word embedding models trained on a large Portuguese corpus, including both Brazilian and European variants. We trained 31 word embedding models using FastText, GloVe, Wang2Vec and Word…
▽ More
Word embeddings have been found to provide meaningful representations for words in an efficient way; therefore, they have become common in Natural Language Processing sys- tems. In this paper, we evaluated different word embedding models trained on a large Portuguese corpus, including both Brazilian and European variants. We trained 31 word embedding models using FastText, GloVe, Wang2Vec and Word2Vec. We evaluated them intrinsically on syntactic and semantic analogies and extrinsically on POS tagging and sentence semantic similarity tasks. The obtained results suggest that word analogies are not appropriate for word embedding evaluation; task-specific evaluations appear to be a better option.
△ Less
Submitted 20 August, 2017;
originally announced August 2017.