Sequel to Introducing Pearls: Scaling Agent Memory from Local Repos to Distributed Knowledge Meshes

Write the blog post where Pearls adds packaging support via a minimal profile on RO-Crate metadata.

ChatGPT Prompt

When we built Pearls, we chose JSON-LD as the native data model and TerminusDB as the storage engine.

That decision had an unexpected consequence.

Because Pearls already stores its graph as JSON-LD documents, adding support for RO-Crate turned out to be remarkably straightforward.

Today we’re introducing OPAL.

OPAL (Open Package Assembly Language) is the RO-Crate profile used by Pearls for Package interoperability.

That’s a much smaller announcement than it sounds.

We didn’t invent a new interchange format.

We adopted an existing one.

Why RO-Crate?

RO-Crate has become one of the most practical ways to package structured metadata alongside digital artifacts.

It already provides:

• JSON-LD as the underlying representation
• stable identifiers
• provenance
• relationships between entities
• broad tooling across research and data management ecosystems

Those are exactly the properties we wanted when Packages leave a running Pearls graph and become portable artifacts.

What Is OPAL?

OPAL is intentionally tiny.

An OPAL Package is simply an RO-Crate that follows a small set of conventions for representing Pearls objects.

Specifically, an OPAL Package:

• is a valid RO-Crate;
• serializes as ro-crate-metadata.json;
• represents Pearls as JSON-LD entities with stable identifiers;
• preserves provenance using standard RO-Crate and PROV-O terms;
• preserves graph relationships as JSON-LD links; and
• round-trips through Pearls without semantic loss.

Everything else comes directly from RO-Crate.

Exporting Packages

Exporting a Package is now as simple as:

pl export --format opal > ro-crate-metadata.json

Importing works the same way:

pl import ro-crate-metadata.json

The result is an ordinary RO-Crate that can be archived, exchanged, version controlled, or consumed by other software.

Why a Profile?

Pearls has its own internal graph model optimized for live, distributed coordination.

RO-Crate is optimized for portable interchange.

Those are different jobs.

OPAL is simply the mapping between them.

By defining that mapping as an RO-Crate profile instead of a new file format, Pearls gains immediate interoperability with existing JSON-LD and RO-Crate tooling while remaining free to evolve its internal implementation.

Looking Ahead

Today, OPAL ships as the reference RO-Crate adapter for Pearls.

Our hope is that it becomes useful beyond Pearls as well.

If another system can import and export the same RO-Crate profile, then Packages can move between implementations without either side needing to understand the other’s internal storage model.

That’s exactly the kind of interoperability we were hoping for.

---

Pearls 1.1 Addendum: Stable Global Identifiers

What else does Pearls need to enable true semantic, versus merely syntactic, RO Crate support in OPAL?

ChatGPT Prompt

The introduction of OPAL revealed a requirement that had always been implicit in Pearls.

A distributed mesh cannot function correctly unless every Pearl has a stable identity that is independent of any particular repository, replica, package, or transport format.

Accordingly, Pearls 1.1 promotes every Pearl to a globally identified entity.

Why

Pearls is fundamentally a distributed knowledge mesh. Pearls are replicated, referenced, merged, and published across organizational boundaries.

Without stable global identifiers, two nodes cannot reliably determine whether they are discussing:

• the same Pearl,
• different versions of the same Pearl,
• or entirely unrelated Pearls.

Local identifiers are sufficient for local coordination. They are not sufficient for distributed reasoning.

Stable identifiers provide semantic continuity across the entire lifecycle of a Pearl.

Design

Every Pearl now possesses a globally stable identifier that remains unchanged regardless of:

• replication,
• movement between repositories,
• publication through OPAL,
• import into another Pearl mesh,
• or subsequent revision.

Operational state continues to evolve. Identity does not.

Relationship to OPAL

This change is not required by OPAL.

Rather, the process of designing OPAL made explicit a property that Pearls already required.

An OPAL package now serializes the existing identity of each Pearl directly into JSON-LD using its @id.

No translation layer or package-specific identifier is introduced.

As a result:

• exported Pearls preserve their identity,
• imported Pearls can be recognized without heuristic matching,
• provenance remains attached to the same entity,
• multiple OPAL packages can independently describe the same Pearl.

A Simpler Architecture

This change significantly simplifies the relationship between Pearls and OPAL.Beads Local operational graph ↓ Pearls 1.1 Distributed operational graph + globally stable Pearl identities ↓ Export OPAL An RO-Crate projection of a selected Pearl graph

OPAL is therefore not another storage format, nor another identity system.

It is a standards-based publication format that preserves the identities already maintained by Pearls.

In hindsight, this is exactly where identity belongs: in the distributed mesh itself, not at its publication boundary.