Beyond Embedded AI: Building a True AI-Native Finance Team
JP's Profitable Growth - Vol 07
Hi (👋 ), welcome back to a new edition of Profitable Growth, an operator's newsletter for operators, at the cross-section of AI, operating cultures and financial discipline.
In this week's edition (est. 8 min. read):
Main Article
Quick Hits
Number to Know
AI Without Handcuffs: Unlocking Multi-Modal LLMs for Finance
I disagree with all the noise about Finance organizations being the last ones to adopt AI. The reality is that Finance teams are already using AI. It’s getting increasingly embedded in the tools they rely on daily, from ERP systems to forecasting software, billing, and spend management platforms. Those vendors have been innovating with AI for years, and it has made their offerings more competitive.
However, indirect usage isn’t the same as building an AI-native Finance function. To unlock real leverage, AI must be accessible and integrated into how teams work daily, directly, and safely. I’ve seen companies try to bridge the gap by letting employees expense tools like ChatGPT, Claude, or Perplexity, then pair that with rigid security policies that effectively muzzle meaningful use: don’t share confidential data, upload files, or trust outputs. It’s like handing someone a sports car and telling them not to go over 30 miles an hour. That approach might check a compliance box, but it doesn’t scale, and it certainly doesn’t build capability. Driving an AI-native culture means investing in secure, sanctioned tools that empower employees to actually use AI without handcuffs.
The space is evolving rapidly, and the number of solutions out there is growing exponentially with the promise of significant productivity gains. To build an AI-native organization, I’d recommend deploying multi-modal large language models (LLMs). Multi-modal refers to the model’s ability to process and interpret not only text but also numbers, images, and documents like PDFs and spreadsheets.
A “crawl, walk, run” approach to driving adoption:
A good first operational milestone would be for your teams to master prompting. I would also encourage forming community groups to share learnings and best practices. Consider inviting prompting experts from your or your investors’ networks to share their best practices and accelerate learning.
Your teams will then be able to identify repetition patterns when using the LLMs, know which ones work best for each use case, and explore automation solutions using AI agents for their workflows.
The third stage could be called the “copilot” stage, where the teams embed the AI agents as integral members of their teams, albeit at an intern level and needing supervision. That’s as advanced as it can pragmatically be at the time I’m writing this article.
In addition, data security and integrity of financial reporting are hard requirements for Controllers and CFOs, as they face serious personal consequences if the financials they publish are inaccurate. They can be held civilly or criminally liable,especially if the misstatements are deemed intentional, with U.S. laws like Sarbanes-Oxley carrying fines and prison time. Even unintentional errors can lead to SEC sanctions, shareholder lawsuits, and career-ending reputational damage.
We’re very much still in a “Trust but Verify” mode for the time being, but that doesn’t mean we can’t use technology to do it faster 🙂
In this edition of Profitable Growth, I get a bit more technical and explore the critical considerations and viable options for piloting and deploying secure, confidential multi-modal LLMs explicitly tailored for finance and accounting teams at tech startups. It could also serve as a blueprint for other functions.
🧐 Essential Criteria for Evaluating Solutions
Below, I evaluate four primary categories of solutions, assuming a cross-functional user base of about 25 team members:
1️⃣ Fully Self-hosted Open-source Solution
Deploying open-source LLMs on your infrastructure provides maximum control over data and operations.
Example Stack:
Model: Mistral or Mixtral (open-source multi-modal models)
Framework: LangChain or similar retrieval augmented generation (RAG) tools
Storage: ChromaDB for vector search
Hosting: AWS EC2 GPU instances (g5.xlarge)
RAG: Documents stored on your own secure cloud storage (S3, Azure Blob Storage with server-side encryption).
Benefits:
Complete data control and privacy; no external exposure.
Customizable models to optimize specific finance use cases (report analysis, forecasting, document processing).
Avoid ongoing vendor lock-in or usage-based licensing fees since those are open-source models.
Cost & Considerations:
The initial setup is technically complex; dedicated engineering or external consultancy is likely required.
AWS Infrastructure: EC2 g5.xlarge (GPU instances) typically range between $700-$1350/month.
Additional operational costs (EFS, ECS Fargate): $50–$150/month.
Ongoing operational overhead: regular updates, security patches, and resource monitoring require partial DevOps resources.
Who Should Consider:
Companies prioritizing absolute data sovereignty and willing to invest in upfront technical setup and ongoing management. As your business grows, you will need to provision additional hardware.
2️⃣ Managed Cloud Services with Dedicated Instances
Cloud providers offer managed LLM services with dedicated instances, ensuring data isolation.
Example Service:
Azure OpenAI Service Dedicated Instances or AWS Bedrock (Dedicated Instances). For RAG use the same setup as Option 1.
Benefits:
Easier deployment compared to fully self-hosted solutions. It’s a quick setup with minimal in-house effort.
Vendor-managed infrastructure reduces internal tech overhead.
Providers offer strict data isolation guarantees. Your data isn’t used to train public models.
Cost & Considerations:
Higher monthly costs ($1500–$3000/month). Azure OpenAI Service’s pricing varies based on model and usage. Provisioned Throughput Units (PTUs) offer predictable costs, with discounts available for reserved capacity.
Per-seat expenses range significantly ($60–$120/month per user).
Must trust vendor assurances about confidentiality and security (thorough SLA reviews required).
Who Should Consider:
Startups seeking to balance ease of use, rapid deployment, and robust confidentiality guarantees without internal infrastructure management complexity. The tradeoff would be the dependency on a single vendor, but you could A/B test with different vendors per team.
3️⃣ Commercial SaaS Solutions with Privacy Guarantees
Third-party SaaS providers offer LLM services with enterprise-grade security and confidentiality assurances.
Example Providers:
Anthropic Claude Enterprise and Cohere Enterprise. In this model, financial documents for RAG would typically be hosted on the provider’s cloud infrastructure.
Benefits:
Rapid deployment; minimal technical oversight.
Optimized user interfaces specifically tailored for finance and accounting tasks.
Providers offer explicit legal agreements ensuring no usage of customer data in external training.
Those are enterprise vendors and often include features like SSO, role-based access, and audit logs.
Cost & Considerations:
Typical cost per user ranges from $30–$150/month.
Costs scale linearly, making this expensive as teams grow.
Data hosted externally, requiring strong trust and rigorous vendor assessment.
Who Should Consider:
Companies needing immediate, easy-to-use solutions, comfortable with managed confidentiality assurances from trusted vendors. Third-party servers likely process the data, and you should ensure you have contractual agreements to cover data usage and retention. You will have limited ability to customize models compared to self-hosted solutions.
4️⃣ Hybrid Solutions (Managed Hosting for Open-source Models)
Utilize services that host open-source models on dedicated infrastructure, combining control with reduced operational overhead.
Example Providers:
Hugging Face, Groq, RunPod, and Replicate. For RAG, you would have the option to choose your preferred secure storage method.
Benefits:
Balances advantages of open-source customization with managed service convenience.
Lower complexity in initial setup than fully self-hosted options.
Providers offer isolated infrastructure and data handling guarantees.
Cost & Considerations:
Hosting costs generally fall between $500–$1500/month. RunPod, for example, offers GPU instances starting at $0.17/hour for low-end GPUs and up to $3.99/hour for high-performance options.
Lower operational overhead compared to self-hosted but still demands some ongoing management.
External infrastructure hosting necessitates thorough data protection assurances.
Who Should Consider:
Startups that want control over model selection and transparency but prefer outsourcing infrastructure management. Here again, you should ensure you have contractual agreements covering data usage and retention.
💰 Summary of Cost Estimates
Note: Costs are approximate and based on available data as of May 2025. Actual costs may differ depending on your usage and choices of configurations.
🏆 Which Option is Best?
That will depend on your specific needs and operational context. I put together a few personas for you to consider:
Budget-Conscious Startups: Hybrid solutions would offer a good balance between cost and control, making them suitable for startups with limited budgets.
Startups with In-House Expertise: Fully self-hosted solutions provide maximum control and customization but require technical expertise.
Rapid Deployment Needs: Managed cloud services offer quick setup with enterprise-grade security, which is suitable for startups needing immediate solutions.
Non-Technical Teams: Commercial SaaS solutions provide user-friendly interfaces and support, which are ideal for teams without technical backgrounds.
Selecting the right solution depends on your startup’s specific needs, budget, and technical capabilities. It’s crucial to weigh the benefits and costs of each option to ensure secure, efficient, and effective access to multi-modal LLMs for your finance and accounting teams.
🙌 Cheat-Sheet for CFOs: How to Further Optimize Costs
Get my cheat-sheet for CFOs on how to further optimize hosting costs on my Profitable Growth blog.
♟️ Final Thought
Of course, Microsoft and Google are holding their annual AI conferences this week, and we should expect new announcements. Still, the four scenarios I laid out earlier in this post should remain valid.
You should also be wary of vendor lock-in strategies for this and your broader tech stack. Vendors like Microsoft, Amazon, and Google want you to buy more of their services, making it harder for you to switch down the line.
Quick Hits: What's On My Radar
Trend: ARR/FTE is all the rage right now, with new cohorts of startups able to to reach multi-million dollars in ARR with small team, redefining what best-in-class looks like in terms of efficiency.
Number to Know
2x
That’s how much weight revenue growth should have when you calculate Bessemer’s Rule of X. CEOs and CFOs are always trying to balance growth vs. profitability. I’ve struggled with it myself every startup I worked at. Bessemer’s analysis and subsequent recommendation is grounded in data.
So, your formula should look like this: (Growth Rate x 2) / FCF Margin)
It essentially challenges the Rule of 40, which in their view, gives equal weight to revenue growth and FCF. They have a very good argument that over time, ARR growth compounds, while FCF doesn’t. Also, when it comes to commanding a high multiple, we know the market continues to put more value on growth.
Until next week, 👋
JP