llama.cpp

SBOM Source: ggerganov/llama.cpp generated using microsoft/sbom-tool

RDF Source: Generated using pyspdxtools

llama.cpp Overview

The main goal of llama.cpp is to run the LLaMA model using 4-bit integer quantization on a MacBook. Thus it is written in pure C/C++ without dependencies.

For local installation it is built using make, CMake, or Zig:

Dependencies

All C/C++ called in the makefile are defined in the repo. These include:

ggml.h
ggml-cuda.
llama.h llama-util.h
examples/common.h
build-info.h

From these libraries, standard C/C++ libraries are called.

In addtion to the C/C++ executables, llama.cpp has a python convert.py script to convert model formats. This script relies on a number of python standard-lib packages as well as two packages defined in requirements.txt:

numpy=1.24
sentencepiece==0.1.98

Generated SBOM

Let’s analyze how accurate and complete the SBOM for this project is. First import the KG form of the SBOM as specified in the header above.

kg = kglab.KnowledgeGraph()
kg.load_rdf("../../sboms/rdf/llama.cpp.spdx.rdf.xml", format="xml")

<kglab.kglab.KnowledgeGraph>

Basic Metadata

Let’s start by looking at the overall size of the KG

show_metadata(kg)

Total Triples: 2390
Distinct Entities: 560
Distinct Properties: 29

show_measures(kg)

edges 2390
nodes 577

Already this looks small

Packages

Since we defined the project dependencies above from manual analysis (just looking through the repo) let’s see how the KG reflects this

package_schema(kg)

	property
0	spdx:copyrightText
1	spdx:downloadLocation
2	spdx:externalRef
3	spdx:filesAnalyzed
4	spdx:licenseConcluded
5	spdx:licenseDeclared
6	spdx:licenseInfoFromFiles
7	spdx:name
8	spdx:packageVerificationCode
9	spdx:relationship
10	spdx:supplier
11	spdx:versionInfo
12	rdf:type

packages = get_package_data(kg)
packages

	package	annotations	attributionTexts	checksums	copyrightText	downloadLocation	externalRefs	hasFiles	licenseConcluded	licenseDeclared	licenseInfoFromFiles	name	packageVerificationCode	supplier	versionInfo	relationships
0	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...				NOASSERTION	spdx:noassertion			spdx:noassertion	spdx:noassertion		llama.cpp	_:Nf15934501ff145728ca622dcfcc12ede	Organization: microsoft	master-66874d4	N1899cd52e36f41d7ae90e1708a43beb9, N9aff126d77...
1	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...				NOASSERTION	spdx:noassertion	Nf512fa65faa54a1696aeb787230abf5b		spdx:noassertion	spdx:noassertion		sentencepiece	NaN	NOASSERTION	0.1.98

The generated SBOM only defines two packages.

The overall llama.cpp package - this is a given
A python package sentencepiece from requirements.txt

We are missing many things here:

numpy which is also defined in requirements.txt
All python stdlib packages which are used in convert.py and other python scripts.

It looks like microsoft/sbom-tool does not include stdlib packages

Again, like the previous page, we are missing a good amount of package metadata that is readily available on pypi.

Finally there are no C/C++ dependencies listed in the SBOM, this could be because all are written in the repo, or because the sbom-tool does not recognize them.

Files

If all C/C++ are written as part of the llama.cpp let’s see how to files are represented.

file_schema(kg)

	property
0	spdx:checksum
1	spdx:copyrightText
2	spdx:fileName
3	spdx:licenseConcluded
4	spdx:licenseInfoInFile
5	rdf:type

get_files_data(kg)

	fileID	fileName	licenseInFile	contributors	licenseConcluded	checksum
0	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./.devops/tools.sh	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:N58f6476bf8684f4aa18f48238695fb78
1	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./LICENSE	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:N479c5a72ddfe4f10a81f94bc35a435a7
2	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./.git/logs/HEAD	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:Ncd53a46d0a11430b8b8d93f56cb4542c
3	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./examples/server/README.md	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:N1063cb53ccfe4b3d8a337a7a392b510d
4	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./pocs/CMakeLists.txt	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:N767949591fae429984a39f70816ce767
...	...	...	...	...	...	...
133	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./requirements.txt	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:N85624c0023ef4faf9877c54c58ff9f80
134	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./examples/save-load-state/save-load-state.cpp	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:N9d8b0cc460b8407e8d17e68610440bac
135	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./.git/hooks/push-to-checkout.sample	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:N6d5d8a8b10f04f9c9c2439dbaf35d4af
136	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./examples/quantize-stats/quantize-stats.cpp	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:Nc00587374b314a2a83e8c922afc0d594
137	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	./ggml-opencl.h	spdx:noassertion		http://spdx.org/rdf/terms#noassertion, http://...	_:N53b7bd97bc194c6fae68d33b93fc498b

138 rows × 6 columns

First thing to point out here, is there are no relationships specified with any of these files. It could be important if these files are being used as libraries within the project to specify that as a relationship.

Relationships

Finally let’s look at what relationships are specified in the KG

rels = get_relationship_data(kg)
rels

	element	elementType	relationshipType	relatedElement	relatedElementType
0	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
1	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
2	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
3	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
4	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
...	...	...	...	...	...
135	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
136	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
137	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
138	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
139	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:SpdxDocument	spdx:relationshipType_describes	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package

140 rows × 5 columns

rels.describe()

	element	elementType	relationshipType	relatedElement	relatedElementType
count	140	140	140	140	140
unique	2	2	3	140	2
top	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_contains	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:File
freq	139	139	138	1	138

It looks like most relationshiips are of the spdx:relationshipType_contains type. Let’s filter those out to see what remains.

rels[~rels['relationshipType'].str.contains('spdx:relationshipType_contains')]

	element	elementType	relationshipType	relatedElement	relatedElementType
53	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package	spdx:relationshipType_dependsOn	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package
139	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:SpdxDocument	spdx:relationshipType_describes	<https://spdx.org/spdxdocs/sbom-tool-1.1.1-182...	spdx:Package

Relationship graph visualization

# get the relationship graph to be visualized
graph = visualize_relationship_graph(kg)

# optional: set the physics layout of the network
graph.force_atlas_2based()
graph.set_edge_smooth('dynamic')

# show graph
graph.show("../figs/fig03.relationship_full.html")

../figs/fig03.relationship_full.html

The color of the nodes in the graph refer to the element type in the spdx specification:

display_relationship_graph_legend()

	SPDX Type	Node Color
0	File	Yellow
1	Package	Blue
2	SPDXDocument	Red

Other Elements

Here’s what the KG contains other than Packages, Files, and Relationships

query = """
SELECT DISTINCT ?type
WHERE {
    ?element rdf:type ?type
}
"""
kg.query_as_df(query)

	type
0	spdx:SpdxDocument
1	spdx:Relationship
2	spdx:Package
3	spdx:File
4	spdx:Checksum
5	spdx:ExternalRef
6	spdx:PackageVerificationCode
7	spdx:CreationInfo

This is nothing specific for AI workflows yet