GSoC Week1

Hi,
This week focused on the design and implementation of a dynamic update mechanism for SBO (Systems Biology Ontology) data.

Key Decisions and Progress

1. Data Update Strategy

  • Decision: On application startup → Check for updates → Download new data → Application becomes available

  • Reason: Prioritize data consistency

  • Status: ✅ Implemented and Pull Request submitted

2. User Control Over Updates

  • Draeger’s Suggestion: Allow users to skip updates to support reproducibility of specific versions

  • Nantia’s Suggestion: Enforce updates to ensure data consistency

  • Status:  Further discussion and decision in Wednesday's meeting

3. Data Version Management Strategy

  • System Version History Management

    • strategy1:  Only retain the latest version to save storage 

    • startegy 2:Retain up to 3 versions (2 historical + 1 latest)

    • Status:  Further discussion and decision in Wednesday's meeting

  • User Import History Management

    • strategy：whether or not Support manual loading of local OLS files (suggested by Draeger

    • Status:  Further discussion and decision in Wednesday's meeting

Technical Implementation Progress

Core Feature Development

Key Functions Implemented

  • Version Checking & Update Detection

    • get_remote_updated_time(): Fetch latest "updated"  timestamp from ols api.

    • get_local_updated_time(): Retrieve local SBO "updated" timestamp

    • compare_timestamps(): Determine if update is required

  • Data Download & Processing

    • Full SBO ontology download

    • Paginated term retrieval (100 per page)

    • Fetch parent-child hierarchy relations

    • API Rate Control: 0.05-second delay between requests

  • Structured Data Storage

    • Metadata: Update time, version, total term count

    • Term Details: Description, label, IRI, OBO ID, parent link, parent label, parent OBO ID,

    • Statistical Summary

  • Intelligent File Management

    • Auto-naming based on  "updated"  timestamp

    • Supports co-existence of multiple versions

    • Incremental update support

Technical Highlights

  • Incremental Updates: Download only when updated is detected

  • Fault Tolerance: Detailed error handling and logging

  • API Friendliness: Prevent OLS server overload

  • Data Integrity: Full hierarchical structure information retrieved

Comments

Post a Comment

Popular posts from this blog

GSoC week2

Hi, As discussed in the last blog, I started working on my first milestone of fetching SBO table from OLS to SBO annotator. This blog contains a summary of the meeting with the new features I added and the issues I worked on and fixed this week to add SBML L3V2 support. So, let's quickly dive in summary: 1. Change Detection Mechanism for SBO Terms ❓ Issue : OLS API’s updated field changed, but no observable differences in content. ✅ Observation : Compared OLS (API) and GitHub .owl versions – content is identical . Latest update timestamp on OLS is 2025, but GitHub file was last modified in 2023. New terms are already included in the GitHub version. ✅ Decision : We can rely on GitHub .owl file for consistency and reduced request time. 2. Fetching ‘is_a’ / Parent Info Performance Issue ⏱️ Fetching subclass_of (parent info) via OLS takes ~10 minutes. ⚡ Other fields can be fetched in <1 minute. ✅ Optimization : Parent info should be pulled from ...
Read more

GSoC Week 5

 This week, I mainly focused on diagnosing why adding KEGG still failed to yield finer-grained SBO classifications across 108 models, extending the EC lookup pipeline, and experimenting with additional data sources. Here’s the complete summary of the work update during the week. Root-Cause Investigation Added adviaecnumber() helper to attempt refinement via EC number –driven mapping. Re-ran the enhanced pipeline on 108 models  ，reactions that remained in generic SBO classes were still  missing EC numbers . Result: no additional reactions could be refined to more specific SBO classes. Conclusion: the main blocker is not the mapping rule itself but the absence of EC numbers for those reactions across the currently consulted sources. New Data Sources Added SEED adapter : uses the Solr query interface with strict null filtering to avoid spurious entries. Reactome adapter : combines web parsing with the QuickGO API to perform a multi-step conversion...
Read more