Skip to content

Knowledge Graph Comparison Features #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tekrajchhetri wants to merge 25 commits into
base: main
Choose a base branch
from
Open

Knowledge Graph Comparison Features #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
ba1986d
poetry lock added
tekrajchhetri Dec 16, 2024
f86cb02
Updated cli.py to include the options necessary for comparing the cha…
tekrajchhetri Feb 6, 2025
18d4fa2
example test graphs in jsonld format
tekrajchhetri Feb 6, 2025
14d0b52
removed unused file
tekrajchhetri Feb 6, 2025
1a05507
unit test for testing the conversion
tekrajchhetri Feb 6, 2025
36b8e5a
Delete test_app.py
tekrajchhetri Feb 6, 2025
f0963da
import updated
tekrajchhetri Feb 6, 2025
04d573f
created shared.py and added necessary helper functions to convert the…
tekrajchhetri Feb 6, 2025
0640287
updated cli, removed --compare_type option
tekrajchhetri Feb 6, 2025
5dfc16b
docstring updated
tekrajchhetri Feb 6, 2025
2845b5e
Create test_data.ttl
tekrajchhetri Feb 6, 2025
23253a7
Update description about compare function
tekrajchhetri Feb 6, 2025
57297fa
updated the test files
tekrajchhetri Feb 7, 2025
88ab466
added test files
tekrajchhetri Feb 7, 2025
a9be679
test jsonld file updated
tekrajchhetri Feb 7, 2025
3a0cc55
readme updated to include detailed information
tekrajchhetri Feb 7, 2025
4ef1d27
app.py updated with the necessary functionalities for comparison of t…
tekrajchhetri Feb 7, 2025
a00d433
fixed pyproject.toml to point to correct cli
tekrajchhetri Feb 7, 2025
6ebf166
updated cli to use click.echo
tekrajchhetri Feb 7, 2025
2d5947f
Update poetry.lock
tekrajchhetri Feb 7, 2025
b82bc9d
Update pyproject.toml
tekrajchhetri Feb 7, 2025
0a6f3a9
Jupyter notebook showing the usage of the ProvSense
tekrajchhetri Feb 7, 2025
8b1c2c9
to do item added
tekrajchhetri Feb 7, 2025
013a8c0
DeprecationWarning: ConjunctiveGraph is deprecated, use Dataset inste…
tekrajchhetri Feb 7, 2025
d4172f4
Update Example.ipynb
tekrajchhetri Feb 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
294 changes: 289 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,297 @@ Or get the newest development version via:
pip install git+https://github.com/sensein/ProvSense/.git
```

## Quick start
## Usage
For example files, see `example` directory.

### 1️⃣ Compare Two Knowledge Graph Strings
You can compare two JSON-LD/Turtle/N-Triples formatted knowledge graphs as strings.
```Python
from ProvSense.app import hello_world
from ProvSense.app import compare_items

# KG string comparison
src = """{ "@context": { "ex": "http://example.org/" }, "@id": "ex:Person1", "@type": "ex:Person", "ex:name": "Alice" }"""

dst = """{ "@context": { "ex": "http://example.org/" }, "@id": "ex:Person1", "@type": "ex:Person", "ex:name": "BoB" }"""

# Compare and print results
print(compare_items(src, dst))

```
**Output:**
```python
[
{'subject': 'http://example.org/Person1',
'property': 'http://example.org/name',
'src_value': 'Alice',
'dst_value': 'BoB'},
{'subject': 'http://example.org/Person1',
'property': 'http://example.org/name',
'src_value': 'Alice',
'dst_value': 'BoB'}
]
```

### 2️⃣ Compare Two Knowledge Graph Files
Copy link
Copy Markdown

@yarikoptic yarikoptic Feb 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how specific to "KG" such comparison would be? or could also be applied to PROV graphs?

Thinking of https://github.com/bids-standard/BEP028_BIDSprov/ and their slightly assisted rendering of graphs into pretty images like https://github.com/bids-standard/BEP028_BIDSprov/tree/master/examples/clinica_pipelines . I think it might have been sensible in some cases to provide "Comparison diff" rendering where common nodes would remain gray, added - green, removed - red, and modified - some yet another color ;-)

I wish something like https://ontopea.com/ (random search find) was opensource to tune up for above

Copy link
Copy Markdown
Collaborator Author

@tekrajchhetri tekrajchhetri Feb 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yarikoptic Thanks for the pointer. I checked (https://github.com/bids-standard/BEP028_BIDSprov/tree/master/examples/clinica_pipelines) graph. I can definitely highlight the changes with color when I integrate it with the UI. I will also have a look at those other links in detail.

Pass file paths to compare JSON-LD, TTL, or NT files.

```python
from ProvSense.app import compare_items
print(compare_items("test_src.jsonld", "test_dst.jsonld"))

```
**Output:**
```python
[
{'subject': 'urn:uuid:550e8400-e29b-41d4-a716-446655440000',
'property': 'http://example.org/worksAt',
'src_value': 'http://example.org/Company',
'dst_value': 'http://example.org/CompanyY'},
{'subject': 'urn:uuid:550e8400-e29b-41d4-a716-446655440000',
'property': 'http://example.org/worksAt',
'src_value': 'http://example.org/Company',
'dst_value': 'http://example.org/CompanyY'},
{'subject': 'urn:uuid:550e8400-e29b-41d4-a716-446655440000',
'property': 'http://example.org/location',
'src_value': None,
'dst_value': 'New York'}
]

```
### 3️⃣ Compare Two Directories
Compare all JSON-LD, TTL, and NT files in two folders recursively.

hello_world()
Note: The source and destination folders must contain files with matching filenames for a valid comparison.
```python
from ProvSense.app import compare_items
print(compare_items("src", "dst"))
```

**Output:**
```python

{'test_file.jsonld': [{'subject': 'urn:uuid:550e8400-e29b-41d4-a716-446655440000',
'property': 'http://example.org/worksAt',
'src_value': 'http://example.org/CompanyX',
'dst_value': 'http://example.org/CompanyY'},
{'subject': 'urn:uuid:550e8400-e29b-41d4-a716-446655440000',
'property': 'http://example.org/location',
'src_value': None,
'dst_value': 'New York'},
{'subject': 'urn:uuid:550e8400-e29b-41d4-a716-446655440000',
'property': 'http://example.org/worksAt',
'src_value': 'http://example.org/CompanyX',
'dst_value': 'http://example.org/CompanyY'}],
'test_file_b.jsonld': [{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/Princeton_University',
'dst_value': 'http://dbpedia.org/resource/ETH_Zurich'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/Princeton_University',
'dst_value': 'http://dbpedia.org/resource/University_of_Berlin'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/ETH_Zurich',
'dst_value': 'http://dbpedia.org/resource/Princeton_University'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/ETH_Zurich',
'dst_value': 'http://dbpedia.org/resource/University_of_Berlin'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/Switzerland',
'dst_value': 'http://dbpedia.org/resource/Germany'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/Switzerland',
'dst_value': 'http://dbpedia.org/resource/United_States'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Quantum_mechanics',
'dst_value': 'http://dbpedia.org/resource/Brownian_motion'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Quantum_mechanics',
'dst_value': 'http://dbpedia.org/resource/Theory_of_relativity'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/University_of_Berlin',
'dst_value': 'http://dbpedia.org/resource/Princeton_University'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/University_of_Berlin',
'dst_value': 'http://dbpedia.org/resource/ETH_Zurich'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/sameAs',
'src_value': 'https://www.wikidata.org/wiki/Q937',
'dst_value': 'https://en.wikipedia.org/wiki/Albert_Einstein'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/Germany',
'dst_value': 'http://dbpedia.org/resource/Switzerland'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/Germany',
'dst_value': 'http://dbpedia.org/resource/United_States'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Brownian_motion',
'dst_value': 'http://dbpedia.org/resource/Quantum_mechanics'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Brownian_motion',
'dst_value': 'http://dbpedia.org/resource/Theory_of_relativity'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/alternateName',
'src_value': 'Einstein',
'dst_value': 'Prof. Einstein'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/alternateName',
'src_value': 'Prof. Einstein',
'dst_value': 'Einstein'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Theory_of_relativity',
'dst_value': 'http://dbpedia.org/resource/Quantum_mechanics'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Theory_of_relativity',
'dst_value': 'http://dbpedia.org/resource/Brownian_motion'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/United_States',
'dst_value': 'http://dbpedia.org/resource/Switzerland'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/United_States',
'dst_value': 'http://dbpedia.org/resource/Germany'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/sameAs',
'src_value': 'https://en.wikipedia.org/wiki/Albert_Einstein',
'dst_value': 'https://www.wikidata.org/wiki/Q937'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/ETH_Zurich',
'dst_value': 'http://dbpedia.org/resource/Princeton_University'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/University_of_Berlin',
'dst_value': 'http://dbpedia.org/resource/Princeton_University'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/Princeton_University',
'dst_value': 'http://dbpedia.org/resource/ETH_Zurich'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/University_of_Berlin',
'dst_value': 'http://dbpedia.org/resource/ETH_Zurich'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/Germany',
'dst_value': 'http://dbpedia.org/resource/Switzerland'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/United_States',
'dst_value': 'http://dbpedia.org/resource/Switzerland'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Brownian_motion',
'dst_value': 'http://dbpedia.org/resource/Quantum_mechanics'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Theory_of_relativity',
'dst_value': 'http://dbpedia.org/resource/Quantum_mechanics'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/Princeton_University',
'dst_value': 'http://dbpedia.org/resource/University_of_Berlin'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/affiliation',
'src_value': 'http://dbpedia.org/resource/ETH_Zurich',
'dst_value': 'http://dbpedia.org/resource/University_of_Berlin'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/sameAs',
'src_value': 'https://en.wikipedia.org/wiki/Albert_Einstein',
'dst_value': 'https://www.wikidata.org/wiki/Q937'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/Switzerland',
'dst_value': 'http://dbpedia.org/resource/Germany'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/United_States',
'dst_value': 'http://dbpedia.org/resource/Germany'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Quantum_mechanics',
'dst_value': 'http://dbpedia.org/resource/Brownian_motion'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Theory_of_relativity',
'dst_value': 'http://dbpedia.org/resource/Brownian_motion'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/alternateName',
'src_value': 'Prof. Einstein',
'dst_value': 'Einstein'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/alternateName',
'src_value': 'Einstein',
'dst_value': 'Prof. Einstein'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Quantum_mechanics',
'dst_value': 'http://dbpedia.org/resource/Theory_of_relativity'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/knowsAbout',
'src_value': 'http://dbpedia.org/resource/Brownian_motion',
'dst_value': 'http://dbpedia.org/resource/Theory_of_relativity'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/Switzerland',
'dst_value': 'http://dbpedia.org/resource/United_States'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/nationality',
'src_value': 'http://dbpedia.org/resource/Germany',
'dst_value': 'http://dbpedia.org/resource/United_States'},
{'subject': 'http://dbpedia.org/resource/Albert_Einstein',
'property': 'http://schema.org/sameAs',
'src_value': 'https://www.wikidata.org/wiki/Q937',
'dst_value': 'https://en.wikipedia.org/wiki/Albert_Einstein'},
{'subject': 'http://dbpedia.org/resource/Brownian_motion',
'property': 'http://schema.org/name',
'src_value': 'Brownian',
'dst_value': 'Brownian Motion'},
{'subject': 'http://dbpedia.org/resource/Brownian_motion',
'property': 'http://schema.org/name',
'src_value': 'Brownian',
'dst_value': 'Brownian Motion'},
{'subject': 'http://dbpedia.org/resource/Theory_of_relativity',
'property': 'http://schema.org/name',
'src_value': 'Relativity',
'dst_value': 'Theory of Relativity'},
{'subject': 'http://dbpedia.org/resource/Theory_of_relativity',
'property': 'http://schema.org/name',
'src_value': 'Relativity',
'dst_value': 'Theory of Relativity'}]}
```

## Running Tests

To run all tests in the project:
```sh
python -m unittest discover src/tests
```

To run a specific test file:
```sh
python -m unittest src/tests/test_ttl_conversion.py
```

To run tests with verbose output:
```sh
python -m unittest -v src/tests/test_ttl_conversion.py
```

## To do:
- [ ] A
- [ ] lot
- [ ] Add LLM for advanced comparison
Loading