OpenFoundry

The original pitch was discovery first: describe your use case, compare open models, then tune one. That workflow lived on the hosted product, which is now closed to new users. What survived in code is the deployment side, so expect to choose your model on Hugging Face and bring the name to the CLI.

Deploy from Hugging Face by pasting a full model name such as google-bert/bert-base-uncased, from the SageMaker model catalog by framework, or from your own artifacts through an S3 path or a local folder the tool uploads for you. Three paths, one command, which is how you deploy open source models without touching the console. Custom weights are the interesting case, since that's where most teams get stuck doing it by hand.

Every deploy writes a YAML file describing the model and the endpoint. Pass that file back with --deploy and you get the same stack again, which turns a manual chore into something your CI can run on merge. That's the feature that makes open source AI deployment repeatable rather than a one time experiment.

uvicorn server:app gives you a FastAPI service with two routes: one for endpoint details, one for inference. It accepts temperature, top_k, top_p, max tokens, and repetition penalty, so you can test a model without wiring a client library first. Text models only.

Endpoints run on your AWS account under your SageMaker quota. Nothing routes through OpenFoundry's servers, which settles most procurement and data residency questions early. It also means the meter runs whether you send traffic or not.

The whole tool is MIT licensed with roughly 350 stars on GitHub. You can fork it, patch the deploy path for your own networking rules, or lift the FastAPI wrapper into your own service. No account, no seat fee, no phone call.