Is an agency worth the extra cost over a freelancer?
+
It depends on who would otherwise do the screening, review, and management. If that is you and you are good at it, a freelancer is better value. If it is you and you are not technical, the agency premium is buying something real rather than padding.
When should we make our first in-house hire?
+
Usually once there is enough permanent work to keep someone busy, and once someone can evaluate engineering candidates. Hiring before either is true tends to produce an expensive mistake that takes a year to correct.
Can we mix the three?
+
Most companies past a handful of engineers do, and it is the sensible answer. In-house owns the core and architecture, an agency owns defined areas, freelancers take specified peripheral work. The failure mode is unclear boundaries, not the mix itself.
What is the biggest risk with each?
+
Freelancer: they become unavailable and nobody understands their code. Agency: performance and a messy exit. In-house: a bad hire you cannot easily reverse. All three are manageable, and all three are worth naming before you sign anything.
How do we compare quotes across models fairly?
+
Convert everything to fully-loaded annual cost including your own time, and add the delay before productive work starts. A freelance rate that looks half an agency rate often is not once screening hours and the absence of review are counted.