You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: blogs/agentic-workflows/max-serve-openai-function-calling/README.md
+33-5Lines changed: 33 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,10 @@ In this blog, we will:
10
10
- Showcase how MAX Serve can facilitate local and cloud deployment of LLM-based applications.
11
11
- Walk through a working example that you can clone and run locally.
12
12
13
-
Note that this feature is available in [MAX nightly 25.1.0.dev2025012905](https://github.com/modular/max/commit/2f83e343a4d28a341570b2aab7131ff2da3e19d5) and the code is available in our [GitHub repository](https://github.com/modular/devrel-extras/tree/main/blogs/agentic-workflows/max-serve-openai-function-calling).
13
+
Note that this feature is available in [MAX nightly 25.1.0.dev2025012905](https://github.com/modular/max/commit/2f83e343a4d28a341570b2aab7131ff2da3e19d5) which corresponds to this nightly MAX Serve docker image
The code is available in our [GitHub repository](https://github.com/modular/devrel-extras/tree/main/blogs/agentic-workflows/max-serve-openai-function-calling).
14
17
15
18
To proceed, please make sure to install magic` CLI
16
19
@@ -43,7 +46,14 @@ Function calling allows LLMs to enhance their responses by:
43
46
44
47
## Implementing function calling with OpenAI
45
48
46
-
To illustrate function calling, let's start with a simple example where an AI retrieves the weather using a mock function.
49
+
First things first, you can run the MAX Serve easily on both CPU and GPU as follows:
Then to illustrate function calling, let's start with a simple example where an AI retrieves the weather using a mock function.
47
57
48
58
### `single_function_call.py`
49
59
@@ -104,7 +114,14 @@ Run the example via:
104
114
magic run single_function_call
105
115
```
106
116
107
-
Note that the very first compilation of the model can take a few minutes. The next invocations will be much faster.
117
+
which outputs:
118
+
119
+
```txt
120
+
User message: What's the weather like in San Francisco?
121
+
Weather response: The weather in San Francisco is sunny with a temperature of 72°F
122
+
```
123
+
124
+
**Note** that the very first compilation of the model can take a few minutes. The next invocations will be much faster.
108
125
109
126
### Understanding the function calling format
110
127
@@ -219,7 +236,17 @@ The LLM can now determine when to call `get_weather` or `get_air_quality` based
219
236
Run the example via:
220
237
221
238
```bash
222
-
magic run muli_function_calls
239
+
magic run multi_function_calls
240
+
```
241
+
242
+
which outputs:
243
+
244
+
```txt
245
+
User message: What's the weather like in San Francisco?
246
+
Weather response: The weather in San Francisco is sunny with a temperature of 72°F
247
+
248
+
User message: What's the air quality like in San Francisco?
249
+
Air quality response: The air quality in San Francisco is good with a PM2.5 of 10µg/m³
223
250
```
224
251
225
252
## Deploying with MAX Serve
@@ -437,5 +464,6 @@ OpenAI's function calling and MAX Serve together provide an efficient way to bui
437
464
Now that you've implemented function calling with MAX Serve, you can explore more advanced features and join our developer community. Here are some resources to help you continue your journey:
438
465
439
466
-[Get started with MAX](https://docs.modular.com/max/get-started)
440
-
- Explore [MAX Serve](https://docs.modular.com/max/serve) and [Magic tutorial](https://docs.modular.com/max/tutorials/magic).
467
+
- Explore [MAX Serve](https://docs.modular.com/max/serve) and [MAX Container](https://docs.modular.com/max/container/)
468
+
- Learn more about `magic` CLI in this [Magic tutorial](https://docs.modular.com/max/tutorials/magic)
441
469
- Join our [Discord](https://discord.gg/modular) and our [Modular forum](https://forum.modular.com/)
0 commit comments