Present an LLM agent with a set of strong instruments it could use to work together with a graph database

Information graphs present an ideal illustration of knowledge with versatile knowledge schema that may retailer structured and unstructured data. You need to use Cypher statements to retrieve data from a graph database like Neo4j. One possibility is to make use of LLMs to generate Cypher statements. Whereas that possibility supplies glorious flexibility, the reality is that base LLMs are nonetheless brittle at persistently producing exact Cypher statements. Subsequently, we have to search for a substitute for assure consistency and robustness. What if, as a substitute of growing Cypher statements, the LLM extracts parameters from consumer enter and makes use of predefined capabilities or Cypher templates primarily based on the consumer intent? In brief, you would present the LLM with a set of predefined instruments and directions on when and how you can use them primarily based on the consumer enter, which is often known as the semantic layer.

A semantic layer consists of assorted instruments uncovered to an LLM that it could use to work together with a information graph. They are often of assorted complexity. You possibly can consider every device in a semantic layer as a perform. For instance, check out the next perform.

def get_information(entity: str, kind: str) -> str:
candidates = get_candidates(entity, kind)
if not candidates:
return "No data was discovered in regards to the film or particular person within the database"
elif len(candidates) > 1:
newline = "n"
return (
"Want extra data, which of those "
f"did you imply: {newline + newline.be a part of(str(d) for d in candidates)}"
)
knowledge = graph.question(
description_query, params={"candidate": candidates[0]["candidate"]}
)
return knowledge[0]["context"]

The instruments can have a number of enter parameters, like within the above instance, which lets you implement advanced instruments. Moreover, the workflow can include greater than a database question, permitting you to deal with any edge instances or exceptions as you see match. The benefit is that you simply flip immediate engineering issues, which could work more often than not, into code engineering issues, which work each time precisely as scripted.

Film agent

On this weblog submit, we’ll exhibit how you can implement a semantic layer that permits an LLM agent to work together with a information graph that comprises details about actors, motion pictures, and their rankings.

Taken from the documentation (additionally written by me):

The agent makes use of a number of instruments to work together with the Neo4j graph database successfully.

* Data device: Retrieves knowledge about motion pictures or people, guaranteeing the agent has entry to the newest and most related data.

* Advice Instrument: Gives film suggestions primarily based upon consumer preferences and enter.

* Reminiscence Instrument: Shops details about consumer preferences within the information graph, permitting for a personalised expertise over a number of interactions.

An agent can use data or advice instruments to retrieve data from the database or use the reminiscence device to retailer consumer preferences within the database.
Predefined capabilities and instruments empower the agent to orchestrate intricate consumer experiences, guiding people in the direction of particular targets or delivering tailor-made data that aligns with their present place inside the consumer journey.
This predefined method enhances the robustness of the system by decreasing the creative freedom of an LLM, guaranteeing that responses are extra structured and aligned with predetermined consumer flows, thereby enhancing the general consumer expertise.

The semantic layer backend of a film agent is carried out and accessible as a LangChain template. I’ve used this template to construct a easy streamlit chat software.

Code is out there on GitHub. You can begin the challenge by defining setting variables and executing the next command:

docker-compose up

Graph mannequin

The graph relies on the MovieLens dataset. It comprises details about actors, motion pictures, and 100k consumer rankings of flicks.

The visualization depicts a information graph of people who’ve both acted in or directed a film, which is additional categorized by style. Every film node holds details about its launch date, title, and IMDb score. The graph additionally comprises consumer rankings, which we are able to use to supply suggestions.

You possibly can populate the graph by executing the ingest.py script, which is situated within the root listing of the folder.

Defining instruments

Now, we’ll outline the instruments an agent can use to work together with the information graph. We are going to begin with the data device. Data device is designed to fetch related details about actors, administrators, and flicks. The Python code seems the next:

def get_information(entity: str, kind: str) -> str:
# Use full textual content index to search out related motion pictures or folks
candidates = get_candidates(entity, kind)
if not candidates:
return "No data was discovered in regards to the film or particular person within the database"
elif len(candidates) > 1:
newline = "n"
return (
"Want extra data, which of those "
f"did you imply: {newline + newline.be a part of(str(d) for d in candidates)}"
)
knowledge = graph.question(
description_query, params={"candidate": candidates[0]["candidate"]}
)
return knowledge[0]["context"]

The perform begins by discovering related folks or motion pictures talked about utilizing a full-text index. The full-text index in Neo4j makes use of Lucene beneath the hood. It permits a seamless implementation of textual content distance-based lookups, which permit the consumer to misspell some phrases and nonetheless get outcomes. If no related entities are discovered, we are able to immediately return a response. Alternatively, if a number of candidates are recognized, we are able to information the agent to ask the consumer a follow-up query and be extra particular in regards to the film or particular person they’re inquisitive about. Think about {that a} consumer asks, “Who’s John?”.

print(get_information("John", "particular person"))
# Want extra data, which of those did you imply: 
# {'candidate': 'John Lodge', 'label': 'Particular person'}
# {'candidate': 'John Warren', 'label': 'Particular person'}
# {'candidate': 'John Grey', 'label': 'Particular person'}

On this case, the device informs the agent that it wants extra data. With easy immediate engineering, we are able to steer the agent to ask the consumer a follow-up query. Suppose the consumer is restricted sufficient, which permits the device to establish a selected film or an individual. In that case, we use a parametrized Cypher assertion to retrieve related data.

print(get_information("Keanu Reeves", "particular person"))
# kind:Actor
# title: Keanu Reeves
# yr: 
# ACTED_IN: Matrix Reloaded, The, Aspect by Aspect, Matrix Revolutions, The, Candy November, Replacements, The, Hardball, Matrix, The, Constantine, Invoice & Ted's Bogus Journey, Road Kings, Lake Home, The, Chain Response, Stroll within the Clouds, A, Little Buddha, Invoice & Ted's Wonderful Journey, The Satan's Advocate, Johnny Mnemonic, Pace, Feeling Minnesota, The Neon Demon, 47 Ronin, Henry's Crime, Day the Earth Stood Nonetheless, The, John Wick, River's Edge, Man of Tai Chi, Dracula (Bram Stoker's Dracula), Level Break, My Personal Non-public Idaho, Scanner Darkly, A, One thing's Gotta Give, Watcher, The, Reward, The
# DIRECTED: Man of Tai Chi

With this data, the agent can reply many of the questions that concern Keanu Reeves.

Now, let’s information the agent on using this device successfully. Fortuitously, with LangChain, the method is easy and environment friendly. First, we outline the enter parameters of the perform utilizing a Pydantic object.

class InformationInput(BaseModel):
entity: str = Discipline(description="film or an individual talked about within the query")
entity_type: str = Discipline(
description="kind of the entity. Accessible choices are 'film' or 'particular person'"
)

Right here, we describe that each entity and entity_type parameters are strings. The entity parameter enter is outlined because the film or an individual talked about within the query. Alternatively, with the entity_type, we additionally present accessible choices. When coping with low cardinalities, that means when there’s a small variety of distinct values, we are able to present accessible choices on to an LLM in order that it could use legitimate inputs. As we noticed earlier than, we use a full-text index to disambiguate motion pictures or folks as there are too many values to supply immediately within the immediate.

Now let’s put all of it collectively in a Data device definition.

class InformationTool(BaseTool):
identify = "Data"
description = (
"helpful for when you want to reply questions on varied actors or motion pictures"
)
args_schema: Sort[BaseModel] = InformationInput
def _run(
self,
entity: str,
entity_type: str,
run_manager: Elective[CallbackManagerForToolRun] = None,
) -> str:
"""Use the device."""
return get_information(entity, entity_type)

Correct and concise device definitions are an vital a part of a semantic layer, in order that an agent can appropriately decide related instruments when wanted.

The advice device is barely extra advanced.

def recommend_movie(film: Elective[str] = None, style: Elective[str] = None) -> str:
"""
Recommends motion pictures primarily based on consumer's historical past and desire
for a particular film and/or style.
Returns:
str: A string containing a listing of really useful motion pictures, or an error message.
"""
user_id = get_user_id()
params = {"user_id": user_id, "style": style}
if not film and never style:
# Attempt to suggest a film primarily based on the data within the db
response = graph.question(recommendation_query_db_history, params)
attempt:
return ", ".be a part of([el["movie"] for el in response])
besides Exception:
return "Are you able to inform us about a few of the motion pictures you appreciated?"
if not film and style:
# Advocate prime voted motion pictures within the style the consumer have not seen earlier than
response = graph.question(recommendation_query_genre, params)
attempt:
return ", ".be a part of([el["movie"] for el in response])
besides Exception:
return "One thing went unsuitable"
candidates = get_candidates(film, "film")
if not candidates:
return "The film you talked about wasn't discovered within the database"
params["movieTitles"] = [el["candidate"] for el in candidates]
question = recommendation_query_movie(bool(style))
response = graph.question(question, params)
attempt:
return ", ".be a part of([el["movie"] for el in response])
besides Exception:
return "One thing went unsuitable"

The very first thing to note is that each enter parameters are non-compulsory. Subsequently, we have to introduce workflows that deal with all of the potential combos of enter parameters and the shortage of them. To provide customized suggestions, we first get a user_id , which is then handed into downstream Cypher advice statements.

Equally as earlier than, we have to current the enter of the perform to the agent.

class RecommenderInput(BaseModel):
film: Elective[str] = Discipline(description="film used for advice")
style: Elective[str] = Discipline(
description=(
"style used for advice. Accessible choices are:" f"{all_genres}"
)
)

Since solely 20 accessible genres exist, we offer their values as a part of the immediate. For film disambiguation, we once more use a full-text index inside the perform. As earlier than, we end with the device definition to tell the LLM when to make use of it.

class RecommenderTool(BaseTool):
identify = "Recommender"
description = "helpful for when you want to suggest a film"
args_schema: Sort[BaseModel] = RecommenderInput
def _run(
self,
film: Elective[str] = None,
style: Elective[str] = None,
run_manager: Elective[CallbackManagerForToolRun] = None,
) -> str:
"""Use the device."""
return recommend_movie(film, style)

Thus far, we have now outlined two instruments to retrieve knowledge from the database. Nevertheless, the data move doesn’t need to be one-way. For instance, when a consumer informs the agent they’ve already watched a film and possibly appreciated it, we are able to retailer that data within the database and use it in additional suggestions. Right here is the place the reminiscence device turns out to be useful.

def store_movie_rating(film: str, score: int):
user_id = get_user_id()
candidates = get_candidates(film, "film")
if not candidates:
return "This film is just not in our database"
response = graph.question(
store_rating_query,
params={"user_id": user_id, "candidates": candidates, "score": score},
)
attempt:
return response[0]["response"]
besides Exception as e:
print(e)
return "One thing went unsuitable"
class MemoryInput(BaseModel):
film: str = Discipline(description="film the consumer appreciated")
score: int = Discipline(
description=(
"Score from 1 to five, the place one represents heavy dislike "
"and 5 symbolize the consumer beloved the film"
)
)

The reminiscence device has two obligatory enter parameters that outline the film and its score. It’s an easy device. One factor I ought to point out is that I observed in my testing that it in all probability is smart to supply examples of when to offer a particular score, because the LLM isn’t the very best at it out of the field.

Agent

Let’s put it now all collectively utilizing LangChain expression language (LCEL) to outline an agent.

llm = ChatOpenAI(temperature=0, mannequin="gpt-4", streaming=True)
instruments = [InformationTool(), RecommenderTool(), MemoryTool()]
llm_with_tools = llm.bind(capabilities=[format_tool_to_openai_function(t) for t in tools])
immediate = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful assistant that finds information about movies "
" and recommends them. If tools require follow up questions, "
"make sure to ask the user for clarification. Make sure to include any "
"available options that need to be clarified in the follow up questions "
"Do only the things the user specifically requested. ",
),
MessagesPlaceholder(variable_name="chat_history"),
("user", "{input}"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
]
)
agent = (
{
"enter": lambda x: x["input"],
"chat_history": lambda x: _format_chat_history(x["chat_history"])
if x.get("chat_history")
else [],
"agent_scratchpad": lambda x: format_to_openai_function_messages(
x["intermediate_steps"]
),
}
| immediate
| llm_with_tools
| OpenAIFunctionsAgentOutputParser()
)
agent_executor = AgentExecutor(agent=agent, instruments=instruments, verbose=True).with_types(
input_type=AgentInput, output_type=Output
)

LangChain expression language makes it very handy to outline an agent and expose all its functionalities. We gained’t go into LCEL syntax as that’s past the scope of this weblog submit.

The film agent backend is uncovered as an API endpoint utilizing LangServe.

Streamlit chat software

Now we simply need to implement a streamlit software that connects to the LangServe API endpoint and we’re good to go. We’ll simply take a look at the async perform that’s used to retrieve an agent response.

async def get_agent_response(
enter: str, stream_handler: StreamHandler, chat_history: Elective[List[Tuple]] = []
):
url = "http://api:8080/movie-agent/"
st.session_state["generated"].append("")
remote_runnable = RemoteRunnable(url)
async for chunk in remote_runnable.astream_log(
{"enter": enter, "chat_history": chat_history}
):
log_entry = chunk.ops[0]
worth = log_entry.get("worth")
if isinstance(worth, dict) and isinstance(worth.get("steps"), listing):
for step in worth.get("steps"):
stream_handler.new_status(step["action"].log.strip("n"))
elif isinstance(worth, str):
st.session_state["generated"][-1] += worth
stream_handler.new_token(worth)

The perform get_agent_response is designed to work together with a movie-agent API. It sends a request with the consumer’s enter and chat historical past to the API after which processes the response asynchronously. The perform handles various kinds of responses, updating the stream handler with new statuses and appending the generated textual content to the session state, which permits us to stream outcomes to the consumer.

Let’s now try it out

The ensuing film agent gives a surprisingly good and guided interplay with the consumer.

Conclusion

In conclusion, the mixing of a semantic layer in language mannequin interactions with graph databases, as exemplified by our Film Agent, represents a major leap ahead in enhancing consumer expertise and knowledge interplay effectivity. By shifting the main target from producing arbitrary Cypher statements to using a structured, predefined suite of instruments and capabilities, the semantic layer brings a brand new stage of precision and consistency to language mannequin engagements. This method not solely streamlines the method of extracting related data from information graphs but in addition ensures a extra goal-oriented, user-centric expertise.

The semantic layer acts as a bridge, translating consumer intent into particular, actionable queries that the language mannequin can execute with accuracy and reliability. Because of this, customers profit from a system that not solely understands their queries extra successfully but in addition guides them in the direction of their desired outcomes with higher ease and fewer ambiguity. Moreover, by constraining the language mannequin’s responses inside the parameters of those predefined instruments, we mitigate the dangers of incorrect or irrelevant outputs, thereby enhancing the trustworthiness and reliability of the system.

The code is out there on GitHub.

Dataset

F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: Historical past and Context. ACM Transactions on Interactive Clever Techniques (TiiS) 5, 4: 19:1–19:19. https://doi.org/10.1145/2827872