This content originally appeared on DEV Community and was authored by Alex Aslam
You can feel it in the air, a new kind of pressure. It’s no longer enough to serve dynamic HTML or serialize JSON APIs at scale. The product team wants intelligence. They want the sentiment analysis, the anomaly detection, the personalized recommendation—and they want it now, not in a few hundred milliseconds after a round-trip to some external API.
You’ve built the majestic, sprawling castle of your Rails monolith. It’s a masterpiece of business logic, a kingdom of services and jobs. But now, they’re asking for magic inside its walls. They’re asking for machine learning.
The first instinct is to reach for the familiar: a microservice. A Python service running FastAPI and PyTorch, containerized, deployed to Kubernetes. It’s a valid path. But it’s also a new distributed system to manage, a new network hop, a new point of failure. It feels… heavy. It feels like building a separate alchemist's tower outside the castle gates when what you need is a flask right there in the throne room.
There is another way. A path that feels like a secret. What if you could perform the magic inside the process? What if the king could utter the incantation himself?
This is the journey to the edge.
Act I: The Summoning - Beyond the Python Wall
The world of ML is written in Python. This is its great wall. Our Ruby realm exists on the other side. For years, the only option was to shout over this wall via HTTP or gRPC.
But then, the standard emerged: ONNX (Open Neural Network Exchange). Think of it not as a tool, but as a lingua franca. It’s the Rosetta Stone for machine learning models. A model trained in PyTorch, TensorFlow, or JAX can be converted into a compact, efficient .onnx file—a universal bytecode for neural networks.
This changes everything. The wall is still there, but we now have a messenger who can pass through it, carrying the distilled essence of the model, leaving the Python-specific incantations behind.
The artwork here is one of translation. The complex, framework-specific training script is the original, passionate sonnet in Italian. The ONNX model is its precise, elegant translation into English. It loses nothing in meaning, but gains universal understandability.
Act II: The Crucible - onnxruntime in the Rails Process
The translated model is useless without an interpreter. This is where onnxruntime enters our story. It’s not a Ruby library; it’s a high-performance, cross-platform C++ inference engine. Our challenge is to invite this powerful C++ spirit into our Ruby VM and bind it to our will.
This is where the art begins. We aren’t writing ML code; we are performing a ritual of integration.
We use a gem like onnxruntime (a Ruby binding) to perform this summoning. It’s a delicate process, a pact between realms:
# The first incantation: Inviting the spirit into our project
bundle add onnxruntime
This gem is a thin veil, a Ruby-shaped interface over the powerful C++ core. It handles the terrifying work of managing memory and tensors for us, presenting a Ruby-friendly facade.
The code to load and run a model is deceptively simple, a quiet hum of power:
# config/initializers/onnx.rb
# The moment of binding. We load the compiled wisdom into memory.
$sentiment_analyzer = OnnxRuntime::Model.new(Rails.root.join("lib", "models", "sentiment_analyzer.onnx"))
# app/services/sentiment_analyzer.rb
class SentimentAnalyzer
def analyze(text)
# The pre-processing: turning language into numbers.
# This is its own subtle art (tokenization, embedding).
input_vector = prepare_input(text)
# The invocation. The whisper to the bound spirit.
# 'input_1' is the model's expected input name; a contract from its training.
results = $sentiment_analyzer.predict({ "input_1" => input_vector })
# The post-processing: interpreting the numerical output.
interpret_results(results.first)
end
private
def prepare_input(text)
# ... Your logic to convert text to a tensor (e.g., using a shared tokenizer)
end
def interpret_results(output_array)
# ... Your logic to map the output numbers to a sentiment score
end
end
This is the heart of the artwork: the seamless fusion. The Rails application, a domain of business objects and requests, now holds within it a silent, potent intelligence. A call to SentimentAnalyzer.new.analyze("I love this product!") doesn't trigger a network call. It triggers a mathematical whirlwind inside your process, returning a result in microseconds.
Act III: The Artistry - A Master's Considerations
A junior developer sees the code above and thinks, "Great, I’m done." A senior developer sees a new world of constraints and artistry.
The Weight of the Spirit: ONNXRuntime is a native extension. It adds memory overhead to your Rails process. This isn't a free lunch. You must profile. Is 50MB per process acceptable for the magic it enables? For most, the answer is a resounding yes, but it must be a conscious decision.
The Ritual of Pre/Post-Processing: The model doesn't understand text. It understands tensors (multi-dimensional arrays). The true artistry lies in the
prepare_inputandinterpret_resultsmethods. You must replicate the exact same preprocessing steps used during the model's training in Python. This requires discipline and communication between the ML engineer and the Rails developer. This is the collaborative artwork.The Threading Loom:
onnxruntimesessions are generally thread-safe but not concurrent per session. The pattern of a global model instance loaded once at boot is perfect for Rails' forking servers (like Puma). Each worker process gets its own copy of the model, and requests within that process can run inferences safely. The binding gem handles the C-level thread-safety for you.The Deployment Crucible: Your Dockerfile now needs the
onnxruntimeC++ libraries. It’s a simple addition, but another piece of the pact.
# Your Dockerfile
FROM ruby:3.2.0
# The incantation to prepare the vessel (install the shared libraries)
RUN apt-get update && apt-get install -y libonnxruntime-dev
# ... rest of your build process
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN bundle install
The Masterpiece: Sovereignty
The final artwork isn't the code snippet. It’s the architectural purity.
You have taken something complex and made it a simple, private method on a PORO. You have collapsed a previously distributed architecture into a single, coherent domain operation.
# This is the masterpiece. No network, no microservices, just pure, intelligent function.
def create
@comment = Post.find(params[:post_id]).comments.build(comment_params)
@comment.sentiment_score = SentimentAnalyzer.new.analyze(@comment.body) # 🪄
if @comment.save
redirect_to @post, notice: 'Comment was successfully created.'
else
render :new
end
end
The user gets a real-time, intelligent experience. Your system is simpler, more robust, and faster. You have not just implemented a feature; you have elevated the capabilities of your entire kingdom.
You are no longer just a web developer. You are an alchemist, and your Rails app is the flask where business logic and machine intelligence are fused into gold.
This content originally appeared on DEV Community and was authored by Alex Aslam
Alex Aslam | Sciencx (2025-08-27T20:32:29+00:00) The Alchemist’s Flask: Running ONNX Models in the Rails Crucible. Retrieved from https://www.scien.cx/2025/08/27/the-alchemists-flask-running-onnx-models-in-the-rails-crucible/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.