Editor's note: A few days ago, someone counted the deep learning frameworks used by ICLR paper recruits over the years and found that although TensorFlow is still at the top of the list, PyTorch's usage data in the past year has tripled, which can be comparable to TF. This is surprising news, and many practitioners have begun to face up to this development trend and prepare for the "double repair" issue. In the following, Lunzhi brings readers a simple and easy-to-use PyTorch library released by fast.ai-readers who are interested in PyTorch may wish to try this library first.
With the deep integration of the Internet and knowledge dissemination, online courses are not new to many people. In the field of deep learning, there are three MOOC course platforms most popular with students: Fast.ai, deeplearning.ai/Coursera and Udacity. Among them, because of Jeremy Howard's unique teaching style of simplifying complex and combat-oriented, Fast.ai has always given people the impression of being "down-to-earth":
Research how to apply the most advanced deep learning to practical problems quickly and reliably.
The Fast.ai library is provided, which is not only a toolkit for novices to quickly build deep learning implementations, but also a powerful and convenient resource for providing best practices.
The content of the course is simple and easy to understand, so that as many people as possible can benefit from the research results and software.
During the National Day, Fast.ai released a new, free and open source library for deep learning-fastai. This is a PyTorch library. Although it is still a preview version, it currently provides a consistent API for the most important deep learning applications and data types. Compared with other deep learning libraries, it has significantly improved accuracy and speed. , And the required code is greatly reduced.
Interested developers can visit fastai's GitHub to install: github.com/fastai/fastai/
fastai library
Since the announcement of development last year, after 18 months, the fastai deep learning library v1.0 finally met with you. At the beginning of the project, the developers introduced the advantages of PyTorch as a platform: you can use the flexibility of conventional python code and various functions to build and train neural networks, which can solve a wider range of problems...
Now, through the joint efforts of the Fast.ai team and the PyTorch team, we have ushered in a deep learning library that provides a single and consistent interface for common deep learning applications such as computer vision, text, tabular data, time series, and collaborative filtering. This means that if you have learned to use fastai to create practical computer vision (CV) models, then you can use the same method to create natural language processing (NLP) models, or other models supported by the software.
Early user feedback
Semantic code search on GitHub
Fast.ai courses are an important way for GitHub data scientists and executives (including the CEO) to improve data literacy. Among them, Hithl Husain, a senior machine learning scientist at Github, has been learning deep learning through Fast.ai for the past two years He believes that these MOOC courses have opened a new era of data on Github, making data scientists more confident in solving the latest problems in machine learning.
As the first users of fastai, Hithl Husain and his colleague Ho-Hsiang Wu recently released an experimental version of the tool "Semantic Code Search" that allows developers to find code directly by meaning instead of keyword matching, which means The best search results may not contain the word you searched for. In the official blog post, they introduced their reasons for abandoning Tensorflow Hub and turning to fastai, saying that the latter can more easily access the most advanced architectures (such as AWD LSTMs) and technologies (such as random restart cyclic learning rate).
Semantic code search
For the past 12 months, Husain has been experimenting with the pre-release version of the fastai library. He said:
I chose fast.ai because it can achieve the most advanced technology and innovation with modular and high-level APIs while reducing the amount of calculation while ensuring the same performance. Semantic code search is just the tip of the iceberg. In sales, marketing, and anti-fraud, people can use fastai to bring revolutionary changes to all walks of life.
Generate music
Christine McLeavey Payne is a student who stood out from the previous Fast.ai deep learning course. Her life experience is very rich: from a classical pianist in the San Francisco Symphony Orchestra, to an HPC expert in the financial field, to a neuroscience and medical researcher at Stanford University. Now, she has started another life journey in OpenAI, and in the recent OpenAI project, she used fastai to create an LSTM that can generate piano music and chamber music-Clara.
fastai is an amazing resource. Even a novice like me who is new to deep learning can get the fastai model in just a few lines of code. I don't fully know the principles behind these advanced technologies, but my model can run, and the training time is shorter and the performance is better.
Her music generation model is based on a language model she constructed during class, and using the fastai library's support for the latest NLP technology, she completed the music generation project in just two weeks and achieved good initial results. This is a classic example of the usefulness of the fastai library. With only a few modifications, developers can change the text classification model to a music generation model, which can save a lot of time and effort in practice.
IBM Watson senior researcher's evaluation of music generator Clara
Artistic creation
Architect and investor Miguel Pérez Michaus has been using the pre-release version of fastai for his "Style Reversion" experiment. The so-called "style restoration" is to restore the original image after the style transfer, as shown in the following figure:
Style restoration
He said: "I like to use fastai to create, because it can achieve things that Keras can't achieve, such as generating'non-standard' things." As an early user, he has witnessed fastai's update iterations in the past 12 months:
I was fortunate to experience the A test version of fastai. Although it is only an Alpha version, it fully demonstrated its practicality and flexibility, and allowed me to operate it by people with domain knowledge but no formal computer science background. fastai will get better and better. For the future of deep learning, I personally have a little superficial understanding, that is, we must master the real technical principles behind the black box in detail. In this case, I think fastai will be popular.
Academic Research
In the field of NLP, Polish has always been a challenge because it is a rich language. For example, Polish adjectives change according to the number and gender of nouns. Entrepreneurs Piotr Czapla and Marcin Kardas are the co-founders of deep learning consulting company n-wave. Based on the ideas shown in the course Cutting Edge Deep Learning For Coders, they used fastai to develop a new Polish text classification algorithm. And won the first prize in Poland's top NLP academic competition, and the paper on this new research will be published soon.
According to Czapla, the fastai library is critical to their success:
Fastai is suitable for ordinary people who don't have hundreds of servers. This is one of my favorites. It supports rapid development and prototyping, and incorporates all the best deep learning practices. At the same time, the Fast.ai course is a beacon for me to start learning deep learning. From the day of the class, I started to think about what deep learning can do.
Example: Transfer learning in the field of computer vision
There is a very popular competition on Kaggle: Dogs vs Cats. Participants need to write an algorithm to classify whether the image contains a dog or a cat. This is also a competition often involved in Fast.ai courses, because it represents an important type of problem: transfer learning based on pre-trained models.
We will use this as an example to compare the difference between Keras and fastai in terms of the required code amount, accuracy and speed. The following are all the codes for 2-stage fine-tuning with fastai-not only there is very little code to write, but also very few parameters to set:
data = data_from_imagefolder(Path('data/dogscats'),
ds_tfms=get_transforms(), tfms=imagenet_norm, size=224)
learn = ConvLearner(data, tvm.resnet34, metrics=accuracy)
learn.fit_one_cycle(6)
learn.unfreeze()
learn.fit_one_cycle(4, slice(1e-5,3e-4))
The following table is a comparison of the differences between the two deep learning libraries:
Keras is one of the most popular methods for training neural networks. Although the above data is one-sided, the improvement of fastai can show that Keras is not perfect from the side, and there is still much room for improvement. Regardless of whether it is Keras or other deep learning libraries, to complete the same task, they require much more code than fastai. Correspondingly, their training time will be longer, and the model performance may not be better.
In addition, fastai also has a strong performance in NLP tasks. The following table is a screenshot from the ULMFiT paper, showing the relative error of the text classification algorithm ULMFiT and the top-ranked algorithms in the IMDb dataset:
Summary of text classification performance
Fastai is currently the only library that provides this algorithm. Since the algorithm is built-in, you can directly refer to the Dogs vs Cats code above to reproduce the paper results. Here is how to train the ULMFiT language model:
data = data_from_textcsv(LM_PATH, Tokenizer(), data_func=lm_data)
learn = RNNLearner.language_model(data, drop_mult=0.3,
pretrained_fnames=['lstm_wt103','itos_wt103'])
learn.freeze()
learn.fit_one_cycle(1, 1e-2, moms=(0.8,0.7))
learn.unfreeze()
learn.fit_one_cycle(10, 1e-3, moms=(0.8,0.7), pct_start=0.25)
summary
In the next few months, Fast.ai will successively publish academic papers and blog posts about this deep learning library, including opening a new course. We will continue to pay attention to the progress of the development team. Readers can learn more about PyTorch and experience a different "landscape" while playing TensorFlow.
2835 Led Strip,Single Color Led Strips,Single Color Light,Led Single Light Strip
NINGBO SENTU ART AND CRAFT CO.,LTD. , https://www.lightworld-sentu.com