One of the most important concepts when learning Microsoft Copilot Studio is understanding where an Agent’s answers actually come from.
When we add a file, SharePoint content, or a public website as a Knowledge Source, it is tempting to imagine that every answer produced by the Agent comes directly from those sources.
Understanding Knowledge, Retrieval, Grounding, Instructions, and General Model Knowledge in Microsoft Copilot Studio
A Practical Experiment with a Watch Research Agent
Introduction
One of the most important concepts when learning Microsoft Copilot Studio is understanding where an Agent’s answers actually come from.
When we add a file, SharePoint content, or a public website as a Knowledge Source, it is tempting to imagine that every answer produced by the Agent comes directly from those sources.
However, generative AI systems are more complex.
An Agent can potentially combine configured Knowledge Sources, retrieved information, Instructions, orchestration, and capabilities of the underlying language model when generating an answer.
This laboratory explored an important question:
What happens when we ask an Agent something that is not contained in its configured Knowledge Source?
The experiment also investigated a second question:
Can Instructions constrain the Agent so that it refuses to answer when its configured Knowledge does not support the answer?
The results provide a useful practical introduction to five concepts:
Knowledge → Retrieval → Grounding → Instructions → Answer
while also helping us understand the role of:
General Model Knowledge
1. Laboratory Scenario
We created a Microsoft Copilot Studio Agent called:
WristWatchBuildingCompanion
The Agent was designed as a watch research companion.
Instead of immediately connecting the Agent to a large enterprise Knowledge Source, we intentionally used a small and understandable domain: wristwatches.
This made it easier to determine which information existed in the Knowledge Source and which information did not.
2. The Knowledge File
A Markdown file named:
klgk.md
was uploaded to the Agent as a Knowledge Source.
The file contains a structured collection of watch research resources, including categories such as:
- Watch magazines
- Japanese watch manufacturers
- Swiss manufacturers
- Movement databases
- Horological institutions
- Watch forums
- Market databases
- Auction houses
Examples include Hodinkee, Fratello, Monochrome, Seiko, Citizen, Orient, Rolex, Patek Philippe, Caliber Corner, WatchBase, Phillips, Christie’s, and other resources.
The document also defines a useful research hierarchy:
- Manufacturer archives and original documentation
- Museums and horological institutions
- Contemporary historical documents
- Specialized watch books
- Established watch publications
- Specialist forums and collectors
- Market databases
- Dealer listings, advertisements, and social media
This made the file useful not only as watch information, but also as a controlled Knowledge Source for our experiment.
3. Initial Agent Configuration
At one stage of the experiment, the Agent contained:
Knowledge Sources
klgk.md- Hodinkee as a
Public website
The Copilot Studio interface also displayed suggestions such as:
- Seiko
- Casio
- Citizen
- Rolex
However, these appeared as Suggestions with an Add option.
This distinction was important.
A suggested website displayed by Copilot Studio is not necessarily already configured as an active Knowledge Source.
We therefore distinguished between:
Suggested Knowledge
and:
Configured Knowledge
At the same time:
Web Search = Disabled
This distinction became important during testing.
4. First Question: Patek Philippe Calatrava vs Orient Bambino
We initially asked the Agent to compare:
Patek Philippe Calatrava
and:
Orient Bambino
The Agent generated a detailed comparison discussing subjects such as:
- Dress-watch design
- Historical positioning
- Price positioning
- Mechanical watchmaking
- Design philosophy
- Simplicity
- Elegance
The response was much richer than simply reproducing the content of the Markdown file.
This was our first indication that:
Adding Knowledge does not turn the Agent into a simple document lookup system.
The Agent was generating an answer.
5. Knowledge Is Not a Database Lookup
A useful incorrect mental model would be:
User Question
↓
Agent
↓
Search exact answer in file
↓
Return stored answer
That is not an adequate model for generative AI.
A better conceptual model is:
User Question
↓
Agent
↓
Instructions + Orchestration
↓
Knowledge Retrieval when appropriate
↓
Relevant Context
↓
Grounding
↓
Generative Model
↓
Generated Answer
The final response is generated rather than simply copied from a database record.
6. The AliExpress Experiment
We then deliberately asked a question that went beyond the obvious contents of the Knowledge file.
The question was approximately:
“Is there something similar on AliExpress?”
The Agent responded with several Chinese watch brands and recommendations.
At different stages of our experiments, examples included brands such as:
- Escapement Time
- Baltany
- Addiesdive
- Pagani Design
- Cadisen
This immediately raised an important question:
Where did this information come from?
7. Checking the Knowledge File
The klgk.md file contains many brands and research resources, but it does not contain this specific set of AliExpress recommendations.
Therefore, we could establish an important fact:
Those specific recommendations were not directly supported by the Markdown file.
However, at this stage, another configured Knowledge Source still existed:
Hodinkee
Therefore, the experiment was not yet fully isolated.
We had more than one possible information path.
8. Why Experimental Isolation Matters
When troubleshooting or studying an AI system, changing several variables simultaneously makes it difficult to understand the result.
We therefore adopted the same principle that is useful in traditional software troubleshooting:
Change one variable at a time.
Our goal was to create a controlled configuration where the possible sources of information were reduced.
9. Removing the Public Website
We temporarily removed Hodinkee from the Agent’s configured Knowledge Sources.
We did not add the suggested Seiko, Casio, Citizen, or Rolex websites.
We also kept:
Web Search = Disabled
The resulting configuration was therefore:
Knowledge
klgk.md
Public Website Knowledge
None
Web Search
Disabled
This created a much cleaner experiment.
10. Repeating the AliExpress Question
We then asked again:
“Estou pensando em um relógio parecido com o Patek Calatrava, mas queria algo do AliExpress.”
The Agent still generated recommendations.
It mentioned:
- Escapement Time
- Baltany
- Addiesdive
- Pagani Design
- Cadisen
This was a very important result.
The Agent generated information that was not contained in the configured Markdown Knowledge Source.
At the same time:
- There was no configured Public Website
- Web Search was disabled
Therefore, the Markdown file alone could not explain those specific recommendations.
11. General Model Knowledge
This experiment introduces the concept of:
General Model Knowledge
A Large Language Model should not be imagined as a traditional database containing records such as:
Brand = Baltany
Category = Watch
Marketplace = AliExpress
The model learned statistical relationships and patterns during training.
When generating an answer, it can use capabilities derived from that training to produce information that was not retrieved from the Agent’s configured Knowledge Sources.
A useful conceptual distinction is therefore:
Configured Knowledge
Information explicitly connected to the Agent.
versus:
General Model Knowledge
Capabilities and information represented through the underlying model’s training.
12. Why This Matters
This distinction has major consequences.
Suppose an Agent says:
“This watch is commonly available on AliExpress.”
If the Agent did not perform a current search, that statement should not automatically be interpreted as:
“The Agent just checked AliExpress and confirmed current availability.”
The generated statement may instead reflect general knowledge or patterns learned by the model.
This distinction becomes even more important for dynamic information such as:
- Current prices
- Inventory
- Product availability
- Current policies
- Current regulations
- Current organizational procedures
- Recently changed documentation
Generative plausibility is not the same thing as verified current evidence.
13. The Festina Test
We also asked the Agent about:
Festina
The Agent generated information about Festina despite the fact that Festina was not part of the Markdown Knowledge Source.
Again, this helped demonstrate that the Agent was capable of generating answers beyond the explicit contents of the configured file.
This became an excellent negative test.
Instead of asking:
“Can the Agent answer something contained in Knowledge?”
we deliberately asked:
“What happens when Knowledge does not contain the answer?”
Negative tests are extremely useful when studying grounding.
14. The Next Question
At this point, we had demonstrated something important:
The Agent could answer questions using information beyond what existed in the configured file.
The next question became:
Can we prevent that behavior?
This introduced another fundamental Copilot Studio concept:
Instructions
15. Adding Restrictive Instructions
We modified the Agent’s Instructions with the following behavior:
“Answer questions using only the information available in the configured Knowledge Sources. Do not use general knowledge to fill missing information. If the answer cannot be supported by the configured Knowledge Sources, explicitly say that the available Knowledge does not contain enough information to answer the question.”
The purpose of this Instruction was not to add information.
It was to change the Agent’s behavior.
This distinction is fundamental:
Knowledge provides information.
Instructions influence behavior.
16. Repeating Exactly the Same Question
After changing the Instructions, we repeated essentially the same AliExpress question:
“Estou pensando em um relógio parecido com o Patek Calatrava, mas queria algo do AliExpress.”
This time, the Agent behaved very differently.
Instead of recommending Baltany, Pagani Design, Cadisen, or other brands, it responded that the available sources did not contain information about specific AliExpress models similar to the Patek Philippe Calatrava.
It explicitly stated that it could not provide specific recommendations without extrapolating beyond the supplied material.
This was exactly the behavior we wanted to observe.
17. The Reference Was Also Visible
The Copilot Studio test interface displayed:
1 reference
and identified:
klgk.md
This was an especially useful observation.
The response was not simply refusing arbitrarily.
The interface showed that the configured Markdown file participated as a reference in the answer.
The file contains Patek Philippe as one of the important Swiss manufacturers and research sources, while it does not contain the requested AliExpress recommendations.
The Agent therefore had enough grounded context to discuss the subject at a limited level, but not enough configured Knowledge to produce the requested product recommendations while respecting the new Instructions.
18. Before and After
The experiment can be summarized as follows.
| Configuration | Result |
|---|---|
klgk.md available | Agent has configured watch Knowledge |
| General answering behavior | Agent recommended AliExpress-related brands |
| Public website removed | Recommendations still occurred |
| Web Search disabled | Recommendations still occurred |
| Restrictive Instructions added | Agent stopped extrapolating |
| Same question repeated | Agent said Knowledge was insufficient |
| Reference inspection | klgk.md appeared as the reference |
This is a very useful practical demonstration of the interaction between Knowledge and Instructions.
19. Knowledge
We can now define the first component more clearly.
Knowledge answers:
“What information sources are available to the Agent?”
Examples in an enterprise environment could include:
- SharePoint sites
- SharePoint documents
- Uploaded files
- Public websites
- Dataverse data
- Other supported enterprise sources
In our experiment:
Knowledge Source = klgk.md
Knowledge does not necessarily determine every word the model is capable of generating.
It provides information that can be retrieved and used to support the Agent.
20. Retrieval
The next concept is:
Retrieval
Retrieval answers:
“Which information from the available Knowledge is relevant to the current request?”
If the Knowledge Source contains hundreds or thousands of pieces of information, the entire source does not need to become the context for every answer.
Conceptually:
Knowledge Source
↓
User Question
↓
Search / Retrieval
↓
Relevant Information
The retrieved information can then become part of the context used by the Agent.
21. Grounding
The next concept is:
Grounding
A useful practical definition is:
Grounding is the use of retrieved information as evidence and context for the generated response.
Conceptually:
Knowledge
↓
Retrieval
↓
Relevant information
↓
Grounding
↓
Generated Answer
Grounding helps connect generative behavior to trusted or selected information.
This becomes extremely important in enterprise scenarios.
22. Instructions
Instructions solve a different problem.
They answer:
“How should the Agent behave?”
For example:
Knowledge may contain a corporate policy.
Instructions can say:
- Use only approved policies.
- Do not invent missing procedures.
- Clearly state when information is unavailable.
- Keep answers concise.
- Ask for clarification when necessary.
- Do not extrapolate beyond approved documentation.
Therefore:
Knowledge ≠ Instructions
Knowledge provides content.
Instructions provide behavioral guidance.
Our experiment demonstrated this distinction very clearly.
23. General Model Knowledge
General Model Knowledge answers another question:
“What can the underlying model generate based on what it learned during training?”
Before we added the restrictive Instruction, the Agent demonstrated that it could produce watch-related information beyond the explicit contents of klgk.md.
This does not mean that the Agent performed a live search.
It also does not mean that there is necessarily a conventional internal database being queried.
It means that the underlying generative model has learned relationships that can participate in answer generation.
24. The Complete Mental Model
Our simplified mental model can now evolve from:
User
↓
Agent
↓
Answer
into:
User Prompt
↓
Agent
↓
Instructions
↓
Orchestration
↓
Knowledge Sources
↓
Retrieval
↓
Relevant Context
↓
Grounding
↓
Generative Model
↓
Answer
The generative model also has capabilities derived from its training.
Therefore, without appropriate constraints, an answer may contain information that was not retrieved from the configured Knowledge Source.
25. Grounded Answer vs Plausible Answer
This leads to one of the most important lessons from the laboratory.
There is a difference between:
A plausible answer
and:
A grounded answer
A plausible answer may sound correct because the language model understands the domain.
A grounded answer is supported by information retrieved from the sources we intend the Agent to use.
For casual watch discussions, general model knowledge may be useful.
For enterprise systems, the distinction can be critical.
26. Moving the Same Experiment to SharePoint
Now imagine replacing klgk.md with a SharePoint library containing:
Corporate Policies
A user asks:
“Can employees carry unused vacation days into next year?”
Suppose the SharePoint policy does not answer this question.
Without appropriate behavioral constraints, we do not want the Agent to produce a plausible HR policy based on what similar organizations commonly do.
We would prefer:
“The available corporate policy does not contain enough information to answer this question.”
This is essentially the same behavior we just tested using watches.
That is why this apparently simple watch experiment has direct relevance to enterprise Agent architecture.
27. Knowledge Does Not Automatically Mean Exclusivity
Another important lesson is:
Adding a Knowledge Source does not necessarily mean that the Agent becomes conceptually equivalent to a closed-book question-answering system.
We demonstrated this experimentally.
Initially:
klgk.md
↓
Question outside explicit file content
↓
Agent still answered
After behavioral restriction:
klgk.md
↓
Question outside supported content
↓
Agent refused to extrapolate
Therefore, if strict source-grounded behavior is a business requirement, we should explicitly design and test for it rather than merely assuming that adding Knowledge automatically creates that behavior.
28. Instructions Do Not Add Knowledge
This distinction deserves emphasis.
When we added:
“Do not use general knowledge to fill missing information.”
we did not teach the Agent anything new about watches.
The Knowledge Source remained the same.
What changed was:
Agent behavior
Before:
Missing Knowledge
↓
Model fills the gap
↓
Answer
After:
Missing Knowledge
↓
Instruction prevents unsupported completion
↓
Agent reports insufficient information
This is an excellent example of why Instructions are an architectural component rather than merely cosmetic prompt text.
29. A Useful Enterprise Pattern
From this experiment we can derive a useful pattern for enterprise Agents:
Trusted Knowledge
↓
Retrieval
↓
Grounding
↓
Strict Instructions
↓
Generated Answer
If evidence is insufficient:
↓
Explicit uncertainty / refusal to extrapolate
This is often preferable for domains such as:
- HR policies
- Legal procedures
- Compliance documentation
- Security procedures
- Finance
- Corporate governance
- Technical operating procedures
The acceptable degree of generative freedom should depend on the business scenario.
30. Not Every Agent Needs Strict Grounding
The opposite is also important.
Strict grounding is not always desirable.
Consider our watch companion.
If the goal is:
“Help me explore watches, styles, brands, history, and ideas.”
then allowing broader model knowledge can make the Agent considerably more useful.
But if the goal is:
“Answer only from our verified historical watch archive.”
then strict grounding becomes desirable.
Therefore, neither behavior is universally correct.
The architecture should follow the requirement.
31. Two Different Agent Personalities
We could conceptually create two watch Agents.
Exploratory Watch Companion
Knowledge:
Curated watch sources
Behavior:
Can supplement information with general model knowledge.
Purpose:
Discovery, conversation, ideas, comparisons.
Curated Watch Research Agent
Knowledge:
Approved historical sources
Behavior:
Only makes factual claims supported by configured Knowledge.
Purpose:
Research and controlled documentation.
Both are valid.
They solve different problems.
32. Security and Governance Implications
The same distinction becomes even more important in Microsoft 365.
Imagine an Agent connected to SharePoint.
The Agent must not only answer:
“What information do I know?”
We must also ask:
“What information am I allowed to use?”
and:
“What information am I allowed to infer?”
This introduces governance concerns involving:
- Permissions
- Authentication
- Data access
- Knowledge boundaries
- Instructions
- Source quality
- Data sensitivity
- Hallucination risk
- Monitoring
- Testing
Grounding is therefore not simply a search feature.
It participates in the trust model of an enterprise Agent.
33. What This Experiment Did Not Prove
It is equally important to document the limits of the experiment.
We demonstrated that the Agent produced information not present in our remaining Markdown Knowledge Source before restrictive Instructions were added.
We also demonstrated that changing the Instructions caused the Agent to stop producing those unsupported recommendations.
However, this experiment alone does not expose every internal decision made by the Copilot Studio orchestration or underlying model.
We should therefore avoid claiming that we observed the model’s internal reasoning.
What we observed was its external behavior under controlled configuration changes.
That distinction is important in technical documentation.
34. Experimental Method Used
The method used in this laboratory was simple but valuable:
1. Establish a known Knowledge Source
klgk.md
2. Know what information exists in that source
Watch research websites and related descriptions.
3. Ask about information outside the source
AliExpress recommendations.
4. Observe the answer
The Agent generated recommendations.
5. Remove competing Knowledge Sources
Hodinkee was temporarily removed.
6. Keep Web Search disabled
This reduced another possible information path.
7. Repeat the same question
The Agent still generated external recommendations.
8. Change only Instructions
Require answers to be supported by configured Knowledge.
9. Repeat the same question again
The Agent refused to extrapolate.
10. Inspect references
klgk.md appeared as the reference.
This is a good example of controlled troubleshooting applied to generative AI.
35. Key Architectural Distinctions
| Concept | Question it answers |
|---|---|
| Knowledge | What information sources are available? |
| Retrieval | What information is relevant to this request? |
| Grounding | What retrieved evidence supports the answer? |
| Instructions | How should the Agent behave? |
| General Model Knowledge | What can the model generate from capabilities learned during training? |
| Web Search | Should current public web information be searched when the configured capability allows it? |
| Generated Answer | What response is finally produced? |
Keeping these concepts separate prevents many misunderstandings when designing Agents.
36. The Most Important Result
The most important result of the laboratory can be summarized in two experiments.
Experiment A — General Generative Behavior
Question:
Calatrava-like watch from AliExpress
Configured Knowledge:
klgk.md
Result:
The Agent generated brands not contained in the file.
Conceptually:
Knowledge + model capability → broader generated answer
Experiment B — Strict Knowledge Behavior
Same question.
Same Knowledge.
Additional Instruction:
Only answer from configured Knowledge and do not fill gaps using general knowledge.
Result:
The Agent stated that the available Knowledge did not contain enough information.
Conceptually:
Knowledge + Retrieval + Grounding + restrictive Instructions → constrained answer
That difference is the central lesson of the laboratory.
37. From Watches to Enterprise Agents
The watch scenario gave us a safe and understandable way to explore a concept that will later become critical with SharePoint.
Today:
Watch Agent
Question:
“Recommend something from AliExpress.”
Knowledge does not contain it.
Agent must decide whether to extrapolate.
Tomorrow:
Corporate Policy Agent
Question:
“Can I approve this expense under our policy?”
SharePoint Knowledge does not contain the rule.
Agent must decide whether to extrapolate.
The technical mechanism may appear similar.
The business consequence is completely different.
This is why understanding grounding before building Actions is so important.
An Agent that merely gives an incorrect watch recommendation is inconvenient.
An Agent that invents a corporate policy and then executes an Action based on that invented policy can become a serious architectural problem.
Conclusion
This laboratory demonstrated one of the most important principles in Microsoft Copilot Studio:
Knowledge Sources, Instructions, Retrieval, Grounding, and the generative model are related, but they are not the same thing.
A Knowledge Source provides information that the Agent can retrieve.
Retrieval identifies information relevant to the current request.
Grounding uses retrieved information to support the generated response.
Instructions influence how the Agent should behave with that information.
The underlying model can also possess capabilities derived from its training, which means an Agent may be able to generate information beyond the explicit contents of its configured Knowledge Sources unless its behavior is appropriately constrained.
Our experiment demonstrated this behavior clearly.
With only klgk.md configured and Web Search disabled, the Agent could initially recommend AliExpress-related watch brands that were not present in the file.
After adding restrictive Instructions, the same Agent responded that the available Knowledge did not contain sufficient information and explicitly avoided extrapolating beyond the supplied material.
The Copilot Studio test interface also showed klgk.md as the reference used by the constrained response.
The broader architectural lesson is:
Do not assume that adding trusted Knowledge automatically means every generated statement will be limited to that Knowledge.
Instead, decide explicitly what behavior the business scenario requires, configure the Agent accordingly, and test both:
questions that the Knowledge can answer
and
questions that the Knowledge cannot answer.
For enterprise Agents, the second category may be even more important than the first.
Final Mental Model
Knowledge = Information Sources
↓
Retrieval = Find Relevant Information
↓
Grounding = Use Retrieved Evidence
Instructions = Define Behavioral Boundaries
General Model Capability = Generative Knowledge Learned During Training
↓
Generated Answer
And the principle we demonstrated today is:
When the Knowledge does not contain the answer, the Agent’s behavior depends on how we have designed and constrained it.
That is one of the foundations for building reliable Microsoft Copilot Studio Agents.
edvaldo b. guimarães filho 