From a Side Project to TensorFlow's Core: François Chollet on How Keras Came to Be

摘要

30 秒看懂

François Chollet describes starting Keras in early 2015 as a Python-first, scikit-learn-inspired library for LSTMs and convnets at a time when Caffe dominated deep learning. He explains how it became a multi-backend framework, then joined TensorFlow's core after he moved to work with the TensorFlow team, and why TensorFlow 2.0 lets one framework serve both researchers and data scientists.

1Keras began as a tool for recurrent networksChollet found no reusable open-source LSTM implementation in early 2015, so he built his own and later made it work with convnets in the same library.
2Defining models in Python code was a niche betAt the time, Caffe and other major libraries used static configuration files, while Chollet chose code-defined models.
3Keras was "scikit-learn for neural networks"The `fit` function reduces a complex training loop to one call, and Chollet says the "magic is the point" as long as it is delightful.
4Timing helpedReleased in March 2015, Keras arrived just as the community was growing interested in recurrent networks and NLP, and it grew from there.
5Keras ran on multiple backends before it moved into TensorFlowAfter TensorFlow's release in November 2015, Chollet refactored the code so the same codebase could run on TensorFlow or Theano.
6TensorFlow 2.0 removes the usability-versus-flexibility trade-offYou can subclass models and write your own training loops with eager execution, or use the classic high-level Keras workflow, all in one framework.

主题开源与社区 Deep Learning Open-source ecosystems AI History Software engineering 机器学习基础

全文

按章节阅读完整文稿

Keras started in 2015 as a Python-first library for recurrent networks

Chollet began working on Keras in February 2015, when fewer than about 10,000 people were working on deep learning and tooling was immature. Caffe, written mostly in C++, was the library everyone used, and computer vision with convnets was the most popular subfield. He was personally interested in recurrent neural networks, a niche topic then, and he had used Torch 7 and Theano along with Caffe.

He found no good tool for recurrent networks and no reusable open-source LSTM implementation, so he decided to build his own. The first pitch was a library centered on LSTMs, in Python, with models defined in code. This went against the mainstream, because Caffe and several other big libraries used static configuration files in a format like YAML to define models.

Host Lex Fridman calls it a gutsy move to stick with code when the most popular library used configuration files. Chollet notes that other libraries, such as Torch 7 and Lasagne, also defined models in code, but it was the more niche option. Keras's value proposition at the time was what he believes was the first reusable open-source Keras-style implementation of LSTMs, plus the ability to combine LSTMs and convnets in one library.

Usability, borrowed from scikit-learn, made Keras take off

Chollet had done a lot of work with scikit-learn before deep learning and loved it for its usability, so he drew heavily on it when designing Keras. Fridman describes Keras as scikit-learn for neural networks, and Chollet agrees, pointing to the `fit` function that reduces a complex training loop to a single call.

Some people say that hides a lot of detail, he acknowledges, but that is exactly the point. The magic is "delightful", in a good way. He released Keras in March 2015 when the deep learning community was small but many people were becoming intrigued by recurrent networks and NLP, so an easy-to-use implementation arrived at the right moment and grew from there.

Joining Google and TensorFlow turned a side project into a core API

Chollet joined Google a few months after releasing Keras, on a research team doing image classification, which was unrelated to Keras. Soon after arriving he saw an early internal version of TensorFlow, which struck him as an improved version of Theano, so he knew he had to port Keras to it. As a new Googler he was too busy at first, but TensorFlow's public release in November 2015 was a wake-up call.

In December he got Keras running on TensorFlow, though it was more of a refactoring than a port: he abstracted all backend functionality into one module so the same codebase could run on multiple backends, TensorFlow or Theano. TensorFlow became the default option, easier to use and, he says, much faster especially for recurrent networks, and it eventually overtook Theano.

Through 2016 Keras was still a side project, not his job assignment, even as it gained many users. That changed around October 2016, when Rajat Monga, the lead of TensorFlow, came by and invited him to spend a quarter working with the team. Chollet started integrating the Keras API more tightly into TensorFlow, first as a temporary `tf.keras` version in contrib and later in the core, and he says he never returned to his research work.

TensorFlow 2.0 gives researchers and data scientists one framework

Asked about TensorFlow 2.0, Chollet says he has worked on it for a long time and considers it a delightful product that makes huge progress. Previously, Keras was an easy high-level interface, but anyone wanting a lot of flexibility often found the framework got in the way compared with writing everything from scratch.

In 2.0, he says, you get the usability of the high-level interface and the flexibility of the lower-level one, with a spectrum of workflows in between. You can write everything yourself by subclassing models and writing training loops with eager execution, which is flexible, easy to debug and powerful, and it all integrates with higher-level features such as the classic Keras workflows that are ideal for a data scientist or machine learning engineer.

The same set of APIs now suits profiles from researchers to data scientists and everyone in between, instead of forcing a choice between ease of use and control.