Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clean Raw Data: OpenNTP #18

Closed
4 tasks done
rufuspollock opened this issue Sep 6, 2016 · 2 comments
Closed
4 tasks done

Clean Raw Data: OpenNTP #18

rufuspollock opened this issue Sep 6, 2016 · 2 comments
Milestone
sprint 2 - beta2

Comments

@rufuspollock
Copy link
Contributor

rufuspollock commented Sep 6, 2016 •
edited
Loading

This should run AWS.

  • Fetch and uncompress
  • Sample (?) - maybe at rate 1
  • Clean - filter
  • Parse - put into standard internal format
@rufuspollock rufuspollock modified the milestone: beta-2 Sep 6, 2016
@rufuspollock rufuspollock mentioned this issue Sep 6, 2016
Closed
2 tasks
@rufuspollock rufuspollock modified the milestones: beta-2, sprint 2 - beta2 Sep 6, 2016
@chorsley
Copy link
Contributor

chorsley commented Sep 6, 2016

The following filtering possibilities with NTP data have been observed so far:

  • Amplification factor not taken into account yet, though we do have the byte count of responses to help us here.
  • "stratum" value in NTP data might tell us how intentionally the server is designed to be used for public consumption. A value of 16 is unsynchronised (read: completely unreliable) and possibly unintentionally available.

@chorsley
Copy link
Contributor

Closing as data can be processed via ETLv2. Further issues at #48 (processing issues due to file size) and #49 (ideas for further filtering).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
sprint 2 - beta2
Development

No branches or pull requests
2 participants
@rufuspollock @chorsley