Install, start, and verify
Create a dedicated Python environment for your service on the instance. The model path below points to a local directory already synced to that instance. After starting the service, verify its OpenAI-compatible API from the instance itself.
python3 -m venv ~/mlx-service/.venv
~/mlx-service/.venv/bin/python -m pip install mlx-lm
~/mlx-service/.venv/bin/python -m mlx_lm.server \
--model ~/models/quantized-model \
--host 127.0.0.1 --port 8080
curl http://127.0.0.1:8080/v1/models
Check that the model files are complete, the Python environment works, and the port isn't already in use. Options may change between mlx-lm versions, so check the command help for the version installed on your instance before deploying.
ACCESS / 01
Give your team access
By default, the service listens on 127.0.0.1 only. For a small number of internal users, each team member can create an SSH tunnel. For a shared entry point, use a reverse proxy with authentication, access controls, and TLS. Don't expose an unprotected inference endpoint to the public internet.
ssh -N -L 8080:127.0.0.1:8080 xops-node
STARTUP / 02
Keep it running after restart
Once the foreground command is stable, create a user-level launchd configuration for the service. Set the Python executable in the virtual environment, model path, working directory, and log file. Check the config before loading it into the current user session, then verify the process and endpoint after restarting the instance.
plutil -lint ~/Library/LaunchAgents/com.xcodeops.mlx.plist
launchctl bootstrap gui/$(id -u) \
~/Library/LaunchAgents/com.xcodeops.mlx.plist