Hi i am new in weaviate but i think it could fit mu needs :
i’d like to store WebSemantic data from Solid pods in a vector way .
Here is a tricky example : On a pod, user can store data in their public folder like this one
https://spoggy-test2.solidcommunity.net/public/
if you wget https://spoggy-test2.solidcommunity.net/public/ or if you click on the second icon, you will see the data describing the folder in RDF/turtle format
@prefix : <#>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix stat: <http://www.w3.org/ns/posix/stat#>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
@prefix pub: <>.
@prefix n0: <agenttest/>.
@prefix n1: <agora/>.
@prefix n2: <annonces/>.
@prefix blop: <blop/>.
@prefix bookm: <bookmarks/>.
@prefix br: <brains/>.
@prefix dok: <dokieli/>.
@prefix g: <games/>.
@prefix gou: <gouvernance/>.
@prefix gro: <groups/>.
@prefix hel: <helyfe/>.
@prefix mee: <meeting/>.
@prefix mov: <movies/>.
@prefix neu: <neurones/>.
@prefix pic: <pictures%20and%20sounds/>.
@prefix po: <popock/>.
@prefix s: <salut/>.
@prefix Sh: <Shapes%20and%20Default/>.
@prefix shi: <shighl_test/>.
@prefix souk: <soukai-test/>.
@prefix spo: <spoggy/>.
@prefix zoop: <zoop/>.
@prefix tur: <http://www.w3.org/ns/iana/media-types/text/turtle#>.
@prefix shex: <http://www.w3.org/ns/iana/media-types/text/shex#>.
@prefix pl: <http://www.w3.org/ns/iana/media-types/text/plain#>.
pub:
a ldp:BasicContainer, ldp:Container;
dct:modified "2023-06-07T18:51:55Z"^^xsd:dateTime;
ldp:contains
n0:, n1:, n2:, <blim%20blop.tct>, blop:, <bookmarks.ttl>, bookm:, br:,
dok:, g:, gou:, <group-listing.ttl>, gro:, hel:, <issue.shex>,
<issue1.ttl>, <issue2.ttl>, mee:, mov:, <myvisitedplaces.ttl>, neu:,
pic:, po:, s:, Sh:, shi:, souk:, spo:, <tags.ttl>, <work-groups.ttl>,
zoop:;
stat:mtime 1686163915.428;
stat:size 4096 .
n0:
a ldp:BasicContainer, ldp:Container, ldp:Resource;
dct:modified "2022-03-28T12:34:04Z"^^xsd:dateTime;
stat:mtime 1648470844.852;
stat:size 4096 .
n1:
a ldp:BasicContainer, ldp:Container, ldp:Resource;
dct:modified "2020-10-21T16:22:57Z"^^xsd:dateTime;
stat:mtime 1603297377.141;
stat:size 4096 .
n2:
a ldp:BasicContainer, ldp:Container, ldp:Resource;
dct:modified "2021-03-25T11:03:30Z"^^xsd:dateTime;
stat:mtime 1616670210.007;
stat:size 4096 .
<blim%20blop.tct>
a ldp:Resource;
dct:modified "2021-05-22T19:48:54Z"^^xsd:dateTime;
stat:mtime 1621712934.099;
stat:size 30 .
blop:
a ldp:BasicContainer, ldp:Container, ldp:Resource;
dct:modified "2020-05-15T21:25:05Z"^^xsd:dateTime;
stat:mtime 1589577905.332;
stat:size 4096 .
<bookmarks.ttl>
a tur:Resource, ldp:Resource;
dct:modified "2020-05-09T10:34:06Z"^^xsd:dateTime;
stat:mtime 1589020446.625;
stat:size 778 .
bookm:
a ldp:BasicContainer, ldp:Container, ldp:Resource;
dct:modified "2022-08-18T15:33:45Z"^^xsd:dateTime;
stat:mtime 1660836825.753;
stat:size 4096 .
....
it is possible to get it in jsonld format with wget --header=Accept:application/ld+json https://spoggy-test2.solidcommunity.net/public
that gives me things like
[
{
"@id": "https://spoggy-test2.solidcommunity.net/public/",
"@type": [
"http://www.w3.org/ns/ldp#BasicContainer",
"http://www.w3.org/ns/ldp#Container"
],
"http://purl.org/dc/terms/modified": [
{
"@type": "http://www.w3.org/2001/XMLSchema#dateTime",
"@value": "2023-06-07T18:51:55Z"
}
],
"http://www.w3.org/ns/ldp#contains": [
{
"@id": "https://spoggy-test2.solidcommunity.net/public/agenttest/"
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/agora/"
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/annonces/"
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/blim%2520blop.tct"
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/blop/"
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/bookmarks.ttl"
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/bookmarks/"
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/brains/"
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/dokieli/"
},
{
...
and for each ressource
{
"@id": "https://spoggy-test2.solidcommunity.net/public/blop/",
"@type": [
"http://www.w3.org/ns/ldp#BasicContainer",
"http://www.w3.org/ns/ldp#Container",
"http://www.w3.org/ns/ldp#Resource"
],
"http://purl.org/dc/terms/modified": [
{
"@type": "http://www.w3.org/2001/XMLSchema#dateTime",
"@value": "2020-05-15T21:25:05Z"
}
],
"http://www.w3.org/ns/posix/stat#mtime": [
{
"@type": "http://www.w3.org/2001/XMLSchema#decimal",
"@value": "1589577905.332"
}
],
"http://www.w3.org/ns/posix/stat#size": [
{
"@value": 4096
}
]
},
{
"@id": "https://spoggy-test2.solidcommunity.net/public/bookmarks.ttl",
"@type": [
"http://www.w3.org/ns/iana/media-types/text/turtle#Resource",
"http://www.w3.org/ns/ldp#Resource"
],
"http://purl.org/dc/terms/modified": [
{
"@type": "http://www.w3.org/2001/XMLSchema#dateTime",
"@value": "2020-05-09T10:34:06Z"
}
],
"http://www.w3.org/ns/posix/stat#mtime": [
{
"@type": "http://www.w3.org/2001/XMLSchema#decimal",
"@value": "1589020446.625"
}
],
"http://www.w3.org/ns/posix/stat#size": [
{
"@value": 778
}
]
},
when i want to insert into weaviate , i got things like that
{
"class": "Container",
"creationTimeUnix": 1690789670076,
"id": "2db9931d-3845-49e0-a2df-84323963dc39",
"lastUpdateTimeUnix": 1690789670076,
"properties": {
"contains": "{'@id': 'https://spoggy-test5.solidcommunity.net/public/agora/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/annonces/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/biptest.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/blop.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/bookmarks/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/brains/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/Chat/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/df'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/Fractopia/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/groups/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/hello_new'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/hello.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/hospex.ttl'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/machin.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/mik.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/mikmak.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/milo/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/neurones/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/new.ttl'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/newhello'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/Ola.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/portfolio/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/shighl_test/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/table/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/test_graph/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/test-dokieli'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/test.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/test.txt'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/thing.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/thtr.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/verses/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/website/'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/yep.html'}, {'@id': 'https://spoggy-test5.solidcommunity.net/public/Yop%2520from%2520mobile.html'}",
"modified": "{'@type': 'http://www.w3.org/2001/XMLSchema#dateTime', '@value': '2022-07-07T10:08:10Z'}",
"mtime": "{'@type': 'http://www.w3.org/2001/XMLSchema#decimal', '@value': '1657188490.832'}",
"size": "{'@value': 4096}",
"uuid": "https://spoggy-test5.solidcommunity.net/public/"
},
"vectorWeights": null
}
or
{
"class": "Ressource",
"creationTimeUnix": 1690789687810,
"id": "e16a286d-e287-4636-ade1-0873476544cb",
"lastUpdateTimeUnix": 1690789687810,
"properties": {
"modified": "{'@type': 'http://www.w3.org/2001/XMLSchema#dateTime', '@value': '2021-12-03T13:18:48Z'}",
"mtime": "{'@type': 'http://www.w3.org/2001/XMLSchema#decimal', '@value': '1638537528.775'}",
"size": "{'@value': 391}",
"uuid": "https://spoggy-test5.solidcommunity.net/public/Yop%2520from%2520mobile.html"
},
"vectorWeights": null
}
but it does not create object for each resource and does not use cross reference object that i think is an essential point with weaviate.
the folder could have subfolder like https://spoggy-test2.solidcommunity.net/public/groups/ or resources in ttl format like that i’d like to store in weaviate too
@prefix : <#>.
@prefix terms: <http://purl.org/dc/terms/>.
@prefix bookm: <http://www.w3.org/2002/01/bookmark#>.
@prefix XML: <http://www.w3.org/2001/XMLSchema#>.
@prefix scen: <https://scenaristeur.github.io/>.
@prefix n0: <http://xmlns.com/foaf/0.1/>.
@prefix c: </profile/card#>.
@prefix solid: <https://github.com/solid/>.
<> terms:references <#0.48793858480845187>, <#0.6831110182292663>.
<#0.48793858480845187>
a bookm:Bookmark;
terms:created "2020-05-09T10:34:06.158Z"^^XML:dateTime;
terms:title "Agora";
bookm:recalls scen:agora;
n0:maker c:me.
<#0.6831110182292663>
a bookm:Bookmark;
terms:created "2020-05-09T10:33:35.718Z"^^XML:dateTime;
terms:title "Node solid server";
bookm:recalls solid:node-solid-server;
n0:maker c:me.
Can someone help me to find a better way to store this jsonld.
Eventually with batch, recursive to get subfolder/subressources ?
Thxs