A New Way to Plug a Human Brain into a Computer: via Veins
November 22, 2020
How do I find a good graphic designer for a small-scale job?
November 22, 2020

[P] Cortex 0.22: Deploy models to production

Repo Link: https://github.com/cortexlabs/cortex

We've just released version 0.22 of Cortex, our open source deployment platform, and I wanted to share it here with the community.

If you're unfamiliar, Cortex is a platform that makes it easy to build, deploy, and manage machine learning APIs on AWS (GCP support coming shortly). At a high level, Cortex's core features include:

  • Infrastructure automation. Cortex automates all the underlying infrastructure, including autoscaling, cluster management, cloud services, server configuration, monitoring, and more.
  • Reproducible deployment pipelines. On deploy, Cortex automatically packages models, deploys them to the Cortex cluster as autoscaling APIs, and allows you to version them.
  • GPU/ASIC inference. Cortex supports inference on GPU and Inferentia instances. Additionally, Cortex also supports deployments to Spot instances, which can reduce AWS costs by ~40%.
  • Open platform. Serve models from any framework, integrate with any tool, implement whatever inference logic you'd like—Cortex is designed to be flexible and modular to fit your stack.

With this new version, we've added several new features that I wanted to highlight here:

  • Live reloading. Models can now be updated without spinning down an API.
  • Multi-model caching. Related to live reloading, you can now cache models in memory, allowing you to serve a collection of models that is collectively bigger than what will fit in memory.
  • Python client. Cortex now has an official Python client, which allows you to trigger deployments from within Python scripts (i.e. within a notebook).

If you have any questions or feedback, I'd love to hear it. Additionally, if you want to dig deeper, Cortex has documentation here.

Thank you all for your help so far. Several features/important decisions regarding Cortex have been directly informed by feedback from the r/MachineLearning community, and we really appreciate the support.

submitted by /u/calebkaiser
[link] [comments]


Comments are closed.