Netflix has open sourced Polynote, a polyglot notebook with first-class Scala support, Apache Spark integration, multi-language interoperability including Scala, Python, and SQL, as-you-type autocomplete, and more. The move is aimed at simplfying data science and machine learning workflows.
Polynote allows data scientists and machine learning researchers to integrate JVM-based ML platform — which makes heavy use of Scala — with the Python ecosystem’s machine learning and visualization libraries. According to the tech giant, it has seen substantial adoption among Netflix’s personalization and recommendation teams. Netflix is now integrating it with the rest of its research platform.
For the uninitiated, Polynote comes equipped with IDE-like features such as interactive autocomplete and parameter hints, in-line error highlighting, and a rich text editor with LaTeX support. It encourages reproducible notebooks with its immutable data model. Additionally, the Polynote UI is said to offer insights into the state of the kernel by showing kernel status, highlighting currently-running cell code, and showing currently executing tasks.
Netflix said in the blog post, “At Netflix, we have always felt strongly about sharing with the open source community and believe that Polynote has a great potential to address similar needs outside of Netflix.”
The tech giant also gave a word of caution though: “Polynote is still very much in its infancy, so you may encounter some rough edges. It is also a powerful tool that enables arbitrary code execution (“with great power, comes great responsibility”), so please be cognizant of this when you use it in your environment.”