Build an LLM-OS: Unlock AI Assistants with Memory, Knowledge, and Tools

To run the LLM-OS locally, follow these steps:

1. Create a Python virtual environment to keep your dependencies isolated.
2. Install the necessary packages, including the optional AWS libraries for the FI-Data framework.
3. Install Docker Desktop if you haven't already.
4. Create the LLM-OS codebase using the fi workspace create command, and select the "LLM-OS" template to clone.
5. Export your OpenAI API key, as you'll be using GPT-4 as the language model.
6. Export your Anthropic API key for the research assistant (Exa).
7. Run fi workspace up to start the LLM-OS application, which will create the necessary Docker containers for the database and the LLM-OS application.
8. Open your web browser and go to http://localhost:8501 to access the LLM-OS interface.
9. Enter a username and start interacting with the LLM-OS, which has access to a calculator, file system, web search, and Yahoo Finance.
10. You can also add other assistant team members, such as a Python assistant, data analyst, or investment assistant, as demonstrated in other examples.

To test the LLM-OS, try adding a blog post to the knowledge base and asking it a question, such as "What did Sam Altman wish he knew?". The LLM-OS will search its knowledge base and use retrieval-augmented generation to provide the answer.

You can also test the calculator by asking "What is 10 factorial?", and the LLM-OS will use the calculator to provide the result.

The local setup keeps everything contained within Docker, making it easy to manage and deploy.

To run the LLM-OS on AWS, follow these steps:

1. Export your AWS credentials by installing the AWS CLI and running aws configure.
2. Add your subnet IDs to the workspace_settings.py file.
3. Add a password for your application and database in the workspace_settings.py file.
4. Create your AWS resources by running fir workspace up --prod-infra-aws. This will set up the necessary infrastructure, including security groups, secrets, database instance, load balancers, and ECS cluster.
5. Once the resources are created, you'll get a load balancer DNS that you can use to access your LLM-OS running on AWS.
6. You can also access the LLM-OS API by appending /api to the load balancer DNS.
7. Test the LLM-OS by adding a blog post and asking it questions. You can also try more complex tasks, such as comparing stocks using the Yahoo Finance tools.

Remember to check the Fi Data documentation for more detailed instructions and information on how to customize and extend the LLM-OS.

Now that we have the LLM-OS running on AWS, let's test out its functionality. We'll perform a few tasks to see how the system operates.

First, let's add a blog post to the knowledge base and then ask the LLM-OS a question about the content:

1. Add a new blog post to the knowledge base:

   • The LLM-OS will process the blog post and store the information in the vector database.

2. Ask the question: "What did Sam Altman wish he knew?"

   • The LLM-OS will search its knowledge base, retrieve the relevant information, and use retrieval-augmented generation to provide the answer.

Next, let's test the calculator functionality:

3. Ask the LLM-OS: "What is 10 factorial?"
   • The LLM-OS will use its calculator capabilities to compute the factorial and return the result.

Finally, let's explore the LLM-OS's ability to perform more complex tasks:

4. Ask the LLM-OS to "Write a comparison between NVIDIA and AMD using Yahoo Finance data."
   • The LLM-OS will leverage its access to Yahoo Finance data, as well as its natural language generation capabilities, to provide a comparative analysis of the two companies.

By testing these various functionalities, you can see how the LLM-OS can serve as a powerful AI assistant, capable of accessing and integrating multiple resources to solve complex problems. The seamless integration of the large language model, knowledge base, and external tools demonstrates the potential of this framework for building advanced AI applications.

The llm OS (Large Language Model Operating System) is a powerful framework that allows you to build AI assistants with long-term memory, contextual knowledge, and the ability to take actions using function calling. By integrating the Fi-data framework with the llm OS, you can create a scalable and practical solution for your AI needs.

The key highlights of the llm OS implementation covered in this tutorial are:

1. Leveraging GPT-4 as the Large Language Model: The llm OS uses GPT-4 as the underlying language model, providing advanced natural language processing capabilities.

2. Accessing Software 1.0 Tools: The llm OS gives the AI assistant access to various software tools, such as a calculator, file system, and web search, to enhance its problem-solving abilities.

3. Persistent Memory and Knowledge Storage: The llm OS utilizes a Postgres database and PGVector for storing the AI assistant's memory and knowledge, enabling long-term retention and retrieval.

4. Internet Browsing Capabilities: The AI assistant can browse the internet to gather additional information, expanding its knowledge base.

5. Delegation to Specialized Assistants: The llm OS allows the AI assistant to delegate tasks to other specialized assistants, such as a Python assistant or a data analyst, for more targeted capabilities.

6. Deployment on AWS: The tutorial demonstrates how to deploy the llm OS on AWS, leveraging infrastructure as code to set up the necessary resources, including the database, load balancers, and ECS cluster.

By following the instructions provided in the Fi-data documentation, you can easily set up and run the llm OS locally or on AWS, allowing you to explore the capabilities of this powerful framework and build your own AI assistants.