Broker guide
Open Source AI: What It Means for Client Data
Open-source AI models can run where you choose, but open does not mean private. See how they differ from hosted assistants and when they suit a brokerage.
- Published
- Updated
Open-source AI gives you permission and materials to use, study, change and share an AI system. It can give a brokerage control over where client documents go, but the deployment decides whether those documents stay private.
An open model inside someone else’s chat app still sends your prompts to that operator. A model running entirely inside your own systems can avoid that transfer.
What Open Source AI Means
Open-source AI makes the system available under terms that let others use and modify it. The Open Source Initiative’s definition requires model parameters, the necessary code and enough information about training data to support those freedoms.
Model weights are the numbers learned during training. They help the model produce an answer when it receives a prompt. An open-weight model makes those trained weights available for download, under its particular licence.
That distinction affects what your technology team can inspect or change. Downloadable weights alone don’t establish that the training code and required data information are available. An open-weight licence can also restrict uses or redistribution.
A large language model (LLM) works with language. Some models also accept images, so they can read a scanned page after an application prepares it. A text-only model needs the document’s text extracted first.
ChatGPT is a hosted assistant. OpenAI separately publishes downloadable open-weight models, including gpt-oss-20b. Downloading those weights doesn’t give you a copy of the ChatGPT service.
Open Does Not Mean Private
An open model keeps client data within your chosen boundary only when the surrounding software also keeps it there. The model’s licence doesn’t decide where uploads, chat histories or diagnostic logs are stored.
Consider the same model in different setups.
| Setup | Where client information goes | What the brokerage must control |
|---|---|---|
| Model and document processing on a work computer | The computer processes the files locally | Device access, backups and any software that sends data elsewhere |
| Model on the brokerage’s own server | Files move from staff devices to that server | Staff permissions, server security and retained records |
| Open model in a hosted chat app | The app operator receives the prompts and uploads | The operator’s storage, training-use terms and access rules |
| Open model through an external application programming interface (API) | The provider receives the content sent by the connected application | Provider terms and the application’s own logs and storage |
In a hypothetical example, a broker uploads a payslip to a hosted assistant built on an open model. The weights are downloadable, but the payslip has still left the brokerage. Moving only the model onto an office server won’t fix this if document reading still uses an external service.
The Office of the Australian Information Commissioner (OAIC) recommends avoiding personal information, especially sensitive information, in publicly available generative AI tools. Its guidance on AI products and privacy also covers assessing providers and how they handle inputs.
For a brokerage, open weights are useful when you need control over processing and have someone responsible for operating the system. A managed service can suit a business that needs someone else to run it. Judge that service on its data handling, even when its underlying model is open.
Best Open Source AI Models
For reading scanned documents, choose an image-capable model. For drafting from text already extracted, a text model can do the job without a vision component.
These downloadable models illustrate the choice. Their licences and input types differ, so the label “open” doesn’t make them interchangeable.
| Model version | Licence | Reading documents | Writing use |
|---|---|---|---|
| Qwen3-VL-8B-Instruct | Apache 2.0 | Image and text inputs, with document-structure and optical character recognition capabilities | Drafting answers or summaries from supplied text and page images |
| Mistral Small 3.2, 24B Instruct | Apache 2.0 | Image and text inputs, with document question-answering evaluations in its model card | Instruction-led drafting and chat alongside image reading |
| Gemma 3, 12B instruction-tuned | Gemma Terms of Use | Image and text inputs for questions about document content | Summaries and other text generation under Google’s model terms |
| gpt-oss-20b | Apache 2.0 | Text input only, so a separate process must read scanned pages | Reasoning and drafting from extracted text |
Qwen3-VL-8B-Instruct is a useful starting choice when the task involves scans and page layout. Its model card describes optical character recognition (OCR), which turns visible characters into text.
Mistral Small 3.2 suits a combined image-reading and writing workflow. Gemma 3 is another image-capable option, with its own licence conditions.
For a workflow that already supplies clean text, OpenAI’s gpt-oss-20b documentation identifies a downloadable model with text input and output. It can’t directly read a payslip image.
Model size, available memory and document length affect deployment. A larger model also needs a suitable runtime and enough capacity for the input. For the practical setup choices, see the guide to running a local LLM.
The model’s published capabilities help you choose a type of system. They don’t establish that it will extract every figure correctly from a broker’s documents. A confident summary can still contain a wrong number.
Open Source AI for Document Processing
Open models process business documents as part of a system that reads files, extracts information and returns it for review. Installing a model alone doesn’t create that workflow.
A typical internal process works in this order.
- Staff submit a document through an approved internal application. The application controls who can open the file.
- A parser reads an existing PDF text layer, or OCR reads a scanned page. An image-capable model can also interpret prepared page images.
- The model extracts the requested fields into a fixed format. The instructions tell it to leave missing information empty instead of inventing a value.
- Validation checks compare the output with the source. A reviewer resolves uncertain fields before the information enters a client record.
In a hypothetical payslip workflow, the requested fields include gross pay and pay frequency. A correct amount with the wrong frequency can distort an annual income figure. Keep a page reference beside the extracted value so the reviewer can find it in the original document.
Separate extraction from lending decisions. Reading an income figure doesn’t establish how a particular lender treats that income or whether a loan is suitable.
Before using an internal document workflow with client files, trace one document through every processing service and storage location. Confirm that external OCR, error logs or backups don’t cross the boundary you chose for the model.
