my profile

Abhishek Singh

Graduate Student

I am a big admirer of all scientific disciplines. Among all domains of science, I have a predilection towards Computer Science and Mathematics and that is what I am pursuing right now at the MIT Media Lab. In Computer Science, I had started with doing some interesting projects and internships in Systems and Security and eventually switched to Machine Learning almost three years ago. Right now I am studying and doing research in various aspects of Decentralized AI, this includes Distributed Machine Learning and Privacy Preserving Machine Learning. Before joining MIT, I did my undergraduate study in CS at IIIT Sri City and worked as an intern at a research group in Cisco where I worked on AutoML, Systems, Deep Learning and Security, and Open source contribution in SysML. Apart from Computer Science, I also want to develop a decent level of understanding in all other scientific fields. I am also a strong advocate of environmental and social impact. My days are mostly packed with tasks and deadlines but I try to take out some time for meditation and physical exercise. Occasionally, I also take out time for enjoying thrill-packed movies, learning musical instruments and useless political debates.

personal info

name: Abhishek Singh E-mail: tremblerz@gmail.com
Who am i?
I am a soul in a physical emodiment who is unravelling deep mystical secrets of universe warped in time and hidden in oblivion. Jokes apart, I am a first year graduate student at Camera Culture group at MIT Media Labs with Prof. Ramesh Rasksar as my Advisor.
What I'm really good at?
I think only others can give a fair answer to this question. From my perception of myself, I think I am good at showing grit and persistence, staying unruffled, and learning new or unknown concepts.
How can you contact me?
Shoot me an email or contact me on Twitter, Facebook and Linkedin
What are my hobbies
I have lots and lots of hobbies and the list has been growing even faster than the hype in AI. Recently I have started boarding on the quest of understanding Conciousness. Much of my hobbies are related to Music; ranging from vocals to various instruments. I am trained in Hindustani classical music but have grown rusty since a long time. Currently, I can play Guitar, Synthesizer, harmonium, bit of hand tapping, bongo and started learning violin recently. Going on excursions, tours, skiing and trekking with friends and family is another cluster of my hobby. I also enjoy reading about History, Physics, Biology, Politics and Psychology (to be honest I enjoy reading everything, not just these five).

Research and Patents

Fast Neural Architecture Construction

Fast Neural Architecture Construction

Accepted at NIPS Meta-learning Workshop, Full paper link, related idea's patent pending with US office
project

Fast Neural Architecture Construction is a method to construct deep network architectures by pruning and expansion of a base network. The construction method is, conceivably, analogous to theories of human brain ontogenesis. Instead of constructing a network architecture based on validation accuracy, a single scalar, our method directly compares utility of different network blocks and constructs networks with close to state of the art accuracy, in < 1 GPU day, faster than most of the current neural architecture search methods.

MI-GAN: Malware Image synthesis Using GANs

MI-GAN: Malware Image synthesis Using GANs

Accepted at AAAI Student abstract
project

Majority of the advancement in Deep learning~(DL) has occurred in domains such as computer vision, and natural language processing, where abundant training data is available. A major obstacle in leveraging DL techniques for malware analysis is the lack of sufficiently big, labeled datasets. In this paper, we take the first steps towards building a model which can synthesize labeled dataset of malware images using GAN. Such a model can be utilized to perform data augmentation for training a classifier. Furthermore, the model can be shared publicly for community to reap benefits of dataset without sharing the original dataset. First, we show the underlying idiosyncrasies of malware images and why existing data augmentation techniques as well as traditional GAN training fail to produce quality artificial samples. Next, we propose a new method for training GAN where we explicitly embed prior domain knowledge about the dataset into the training procedure. We show improvements in training stability and sample quality assessed on different metrics. Our experiments show substantial improvement on baselines and promise for using such a generative model for malware visualization systems.

Interpretability of AutoML system

Interpretability of AutoML system

project

In recent years, AutoML has gained a lot of traction in automating machine learning pipeline as it frees up the burden of low-level design, hyperparameter optimization and inductive biases from researchers. Apart from alleviating the problems of manual machine learning pipelines, automated methods of architecture design have shown to outperform all of the bespoke and state of the art neural network architectures designed by experts for certain tasks. This rapid advancement in the automated design of neural network architectures is very promising for deeper insight about novel architectures and knowledge discovery. However, all these network architectures are designed with accuracy as the prime objective and therefore, are highly sophisticated and intricate. Such complicated architectures could be hard to interpret and why certain architectures are preferred over others by these automated methods is unknown. In this work, we present a method of interpreting systems designed for neural architecture search which utilizes information from the sample distribution of automated methods and does not require any modification of the original system. We also use this system to analyze the performance of one of the neural architecture search technique and show a few interesting insights about the technique. The proposed method is generic in design and can be used for a variety of tasks without needing subtle modifications.

Budgeted Neural Architecture Search

Budgeted Neural Architecture Search

Patent filing in progress with Cisco, US
project

We propose a method for performing neural architecture search under a given budget of time and computation resources. Such a method allows leveraging neural architecture search in a practical setting where the user wants to run the search for certain finite time rather than up to convergence which has been assumed by all prior work.

Knowledge Extraction in Neural Architecture Search

Knowledge Extraction in Neural Architecture Search

Patent filing in progress with Cisco, US
project

We employ architecture encoding as a separate module which contains task-specific function and a common embedding space where common knowledge can be shared and hence transfer learning the architecture search process. Our proposed method uses auto-encoder for architecture synthesis which encodes architecture embedding and extracts domain knowledge of different tasks in higher dimensional space of encoder. For obtaining/searching the architecture encoding in a human-usable format we pass it through the decoder. This coupled architecture of encoder-decoder learns to improve encoding and domain knowledge extraction as the training progresses. Example to simplify explanation: if we have three different tasks such as image classification, object detection, and image captioning. The common feature to be learned by all three different architectures is extracting appropriate feature map from a given image sample which requires searching for convolution filters. Hence, common embedding space of search performed on image classification task can benefit object detection and image captioning. Note that task-specific parameters would be still learned from scratch.

AutoML Framework, AMLA

AutoML Framework, AMLA

Accepted at ICML, AutoML Workshop 2018
project

AMLA is an Automatic Machine Learning frAmework for implementing and deployingneural architecture search algorithms. Neural architecture search algorithms are AutoMLalgorithms whose goal is to generate optimal neural network structures for a given task.AMLA is designed to deploy these algorithms at scale and allow comparison of the per-formance of the networks generated by different AutoML algorithms. Its key architecturalfeatures are the decoupling of the network generation from the network evaluation, supportfor network instrumentation, open model specification and a microservices based architec-ture for deployment at scale. In AMLA, AutoML algorithms and training/evaluation codeare written as containerized microservices that can be deployed at scale on a public or pri-vate infrastructure. The microservices communicate via well defined interfaces and modelsare persisted using standard model definition formats, allowing the plug and play of the Au-toML algorithms as well as the AI/ML libraries. This makes it easy to prototype, compare,benchmark and deploy autoML algorithms in production. AMLA is currently being usedto deploy an AutoML algorithm that generates Convolutional Neural Networks (CNNs)used for image classification.

Lightweight Malware detection

Lightweight Malware detection

Pending with US patent office
project

Running high precision models and ensembles on cloud is compute extensive and the traditional solution to the problem involves trad-ing off with accuracy. In majority of the mission critical systemsinvolving security, such a solution is undesirable and risky. To mit-igate this trade-off we introduce an unique architecture which isoriented towards capturing high recall while requiring minimalcompute.By breaking down the problem of widely used one-shot malwaredetection to a two phase detection along with distributing models tothe compute nodes of appropriate capacity we bring down overalllatency substantially.

call me

(+1)--------

Send a message

tremblerz@gmail.com

Visit me

Camera Culture, MIT Media LabsUS

Languages

Hindi : Native Proficiency English : Working Proficiency