Skip to content

Add flow to monitor transect start/end info updates #163

Description

@leewujung

On the Lasker now Kevin Stierhoff has set up a mechanism to update transect_start_end_time.csv that contains the start and end time of individual transect segment. This file will be overwritten each day at 0700, and then usually manually after a new transect or transect segment is processed in Echoview. Therefore, we need to watch the filesystem for update of this file. The watchdog library mentioned in #159 seems also a good candidate for doing this.

What we need for this flow would be:

  • A utility function that watches update for updates of transect_start_end_time.csv
  • Hooking this watchdog event to an event emission in Prefect to trigger a flow run -- this flow will ultimately be the flow that compute NASC for the new transect(s), but for now this can just be a mock flow for testing

Activity

  1. added theissue type on Aug 6, 2026
  2. leewujung commented on Aug 6, 2026

    @leewujung
    MemberAuthor

    For the second part i was curious so did a quick google... Gemini said below haha 😅:

    import time
    from watchdog.observers import Observer
    from watchdog.events import FileSystemEventHandler
    from prefect.deployments import run_deployment
    
    class PrefectTriggerHandler(FileSystemEventHandler):
        def on_created(self, event):
            if event.is_directory:
                return
            
            print(f"New file detected: {event.src_path}. Triggering Prefect...")
            
            # Trigger your Prefect deployment
            run_deployment(
                name="your-flow-name/your-deployment-name",
                parameters={"file_path": event.src_path}
            )
    
    if __name__ == "__main__":
        path_to_watch = "./watched_folder"
        
        event_handler = PrefectTriggerHandler()
        observer = Observer()
        observer.schedule(event_handler, path=path_to_watch, recursive=False)
        
        observer.start()
        try:
            while True:
                time.sleep(1)
        except KeyboardInterrupt:
            observer.stop()
        observer.join()
  3. linked a pull request that will close this issueAdd transect update monitoring flow #174on Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions