Deep learning framework for big data open-sourced by Yahoo

Yahoo has announced that it has open sourced its TensorFlowOnSpark (TFoS) software for distributed deep learning on big-data clusters.

TensorFlowOnSpark is a piece of software created by Yahoo to make the TensorFlow open-source machine learning framework for deep learning compatible with its data sets that sit inside Spark clusters, which some organizations maintain for processing lots of different kinds of data, according to VentureBeat.

About one year back, Yahoo published CaffeOnSpark, an open-source framework that allows distributed deep learning and big-data processing on identical Spark and Hadoop clusters. Yahoo uses CaffeOnSpark in many forms, and the framework has since been enhanced as a result of contributions from the community. It did not, however, address the concerns of those using Google’s popular TensorFlow, which was limited by its inability to be deployed on existing big data clusters.

The newly-released TFoS enables distributed TensorFlow execution on Spark and Hadoop clusters. Using a simple command-line interface and application program interface, the new framework supports all types of TensorFlow programs, direct tensor communication among TensorFlow processes, and scalable training of Inception image classification networks, according to Yahoo.

"Going forward, we will advance TensorFlowOnSpark as we continue to do with CaffeOnSpark," noted Yahoo. "We welcome the community’s continued feedback and contributions to CaffeOnSpark, and are interested in thoughts on ways TensorFlowOnSpark can be enhanced."

View the Yahoo article.

Share your vision-related news by contacting James Carroll, Senior Web Editor, Vision Systems Design

To receive news like this in your inbox, click here.

Join our LinkedIn group | Like us on Facebook | Follow us on Twitter

Learn more: search the Vision Systems Design Buyer's Guide for companies, new products, press releases, and videos