Add options for streams, files and STDIN

Grafmon now supports streaming from sockets, pipes, STDIN,
log files, etc.

README.md

@@ -10,15 +10,19 @@ Monitor metrics in a real-time time-series line-graph
 
 # Features
 
-* Real-time time-series graph of metrics.
-* Provide your own metric command.
-* Several built-in metrics.
-* Polished graph interface.
-* Adjustable refresh rate.
-* Pausable updates.
-* Hide/show lines.
-* Clickable/hoverable lines.
-* Remembers window position/layout.
+* Real-time time-series graph of metrics with several ways to ingest data:
+  * Streaming from file, socket or STDIN
+  * Builtin system monitors
+  * Custom command
+* Polished graph interface
+  * Hover over lines to see data
+  * Click lines to highlight
+  * Pause graph updates
+  * Customize count of ticks along x-axis
+  * Customize count of labels along x-axis
+* Adjustable refresh rate
+  * Defaults to 1000 ms
+* Remembers window position/layout
 
 # Install
 
@@ -27,19 +31,20 @@ Monitor metrics in a real-time time-series line-graph
 # Usage
 
 ```
-usage: grafmon [-h] [-m MONITOR | -c COMMAND] [-r REFRESHRATE] [-t MAXTICKS]
-               [-l MAXLABELS] [--list-monitors]
+usage: grafmon [-h] [-m MONITOR | -c COMMAND | -s STREAM | -f [FILE]] [-r REFRESHRATE] [-t TICKS] [-l LABELS] [--list-monitors]
 
 Monitor metrics in a real-time time-series graph
 
 options:
   -h, --help            show this help message and exit
   -m MONITOR, --monitor MONITOR
-                        Select from builtin monitors to feed data into monitor
-                        [default: pcpu]
+                        Select from builtin monitors to feed data into graph [default: pcpu]
   -c COMMAND, --command COMMAND
-                        User command to feed data into monitor [example: ps
-                        -eo rss,pid,comm --no-headers]
+                        User command to feed data into graph [example: ps -eo rss,pid,comm --no-headers]
+  -s STREAM, --stream STREAM
+                        Streaming user command to feed data into graph [example: tail -f file]
+  -f [FILE], --file [FILE]
+                        Continuously read file or STDIN to feed data into graph
   -r REFRESHRATE, --refreshrate REFRESHRATE
                         Refresh rate in milliseconds [default: 1000]
   -t TICKS, --ticks TICKS
@@ -73,28 +78,83 @@ wchars     Number of bytes written to storage of all processes
 wops       Number of write I/O operations of all processes
 ```
 
-# Custom monitor
+# Custom monitors
 
 Grafmon is not limited to the builtin monitors. Grafmon also accepts user-provided
-commands for monitoring. Grafmon can monitor smart plug metrics to track power usage of
-appliances. Grafmon can monitor IoT sensors and more!
+commands for monitoring. Grafmon can monitor data from files, streams, sockets and STDIN.
+Grafmon can be used to easily monitor IoT sensors such as smart plug metrics to track power
+usage of appliances, and more!
 
-The command must output lines in the format of `<float> <string>` to STDOUT. The `<float>`
-will be the value associated with `<string>`. The `<string>` may contain spaces. Grafmon will
-invoke the command every refresh-rate tick; the command must start and terminate each tick.
-It may be necessary to write a small wrapper script that read chunks of the input at a time
-and terminates within a update tick.
+Demonstration video:
 
-I am planning to implement an `-s, --stream` option to continually read data from an open
-stream, e.g. `grafmon -s 'tail -f log'` or `monitor_cmd | grafmon -` in the near future!
+[![custom monitors video](https://img.youtube.com/vi/sOQtWdZviTY/0.jpg)](https://youtube.com/watch?v=sOQtWdZviTY)
 
-Examples:
+## Data format
+
+The monitor must output lines in the format of `<float> <string>` to STDOUT. The `<float>`
+will be the value associated with `<string>`. The `<string>` may contain spaces.
+
+For example, a monitor for household appliance wattage consumption might output:
+
+     110 Kitchen Refrigerator
+     60 Living Room Television
+     80 Office Computer
+     130 Kitchen Refrigerator
+     65 Living Room Television
+     90 Office Computer
+
+For system processes, it can be helpful to include the PIDs in the `<string>`:
+
+    10 2301 X
+    30 2304 python3
+    50 2250 qemu
+    15 2301 X
+    20 2304 python3
+    60 2250 qemu
+
+## Custom command
+
+     -c COMMAND, --command COMMAND
+
+Grafmon will invoke the command every refresh-rate tick; the command must start and terminate each tick.
+This is ideal for executing short-lived processes to fetch data.
+
+For example:
 
     grafmon -c 'ps -eo pcpu,pid,comm --no-headers'
 
-    grafmon -c 'cat file_regularly_overwritten'
+## Streaming
+
+     -s STREAM, --stream STREAM
+
+Grafmon will invoke the command once and expects the command to remain running. This is ideal for
+fetching data from a long-running process or server.
+
+For example:
+
+     grafmon -s 'tail -f file -n0'
+     grafmon -s 'socat UNIX-LISTEN:data.sock -'
+
+## File or STDIN
+
+     -f [FILE], --file [FILE]
+
+Grafmon will open the file and continuously fetch data. If `[FILE]` is omitted, STDIN
+will be opened.
+
+For example:
+
+     socat TCP-LISTEN:1234 - | grafmon -f
+
+# Customizing ticks and labels
+
+The count of ticks and labels along the x-axis can be customized. The default is 60 ticks
+and 10 labels. This is ideal for the default window size and refresh-rate, providing a graph
+window containing 1 minute's worth of ticks at 1000 ms per update.
+
+If you prefer more or less ticks, use the `-t TICKS, --ticks TICKS` option.
 
-    grafmon -c 'socat TCP-LISTEN:13510 -'
+If you prefer more or less labels, use the `-l LABELS, --labels LABELS` option.
 
 # Notes
 
@@ -117,5 +177,5 @@ be hovered-over to see a tooltip of their values. Their values will also appear
 bar at the bottom of the window.
 
 To zoom the graph in to inspect lower valued lines, you can untick the checkboxes in the list
-on the left. Unticking checkboxes for lines with the highest peaks will hide the lines and
+on the left. Unticking checkboxes for lines with the highest peaks will hide those lines and
 the graph will zoom in to fit the next highest peaks.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "grafmon"
-version = "1.0.1"
+version = "1.1.0"
 dependencies = [
   "pyqtgraph",
   "PyQt6",

src/grafmon/app.py

@@ -4,9 +4,11 @@
 
 from PyQt6.QtWidgets import QApplication
 
-from .context    import Context
-from .mainwindow import MainWindow
-from .timer      import Timer
+from .context     import Context, MonitorType
+from .filemonitor import FileMonitor
+from .mainwindow  import MainWindow
+from .monitor     import Monitor
+from .timer       import Timer
 
 def exec():
     context = Context()
@@ -17,6 +19,11 @@ def exec():
         print("Error: Refresh rate cannot be less than 100 ms.", file=sys.stderr)
         sys.exit(1)
 
+    if context.monitor_type == MonitorType.FILE:
+        context.monitor = FileMonitor(context)
+    else:
+        context.monitor = Monitor(context)
+
     context.app = QApplication(sys.argv)
     w = MainWindow(context)
     w.show()

src/grafmon/context.py

@@ -7,7 +7,9 @@
 
 class MonitorType(Enum):
     BUILTIN = 1
-    USER = 2
+    COMMAND = 2
+    STREAM = 3
+    FILE = 4
 
 class Context:
     def __init__(self,
@@ -22,22 +24,33 @@ def __init__(self,
         self.tick_mod = 5
         self.monitor_type = MonitorType.BUILTIN
         self.selected_width = 4
+        self.fatal_error = 0
 
     def initFromFile(self, path: str):
         pass
 
     def initFromArgs(self, args: list[str] | None = None):
-        parser = argparse.ArgumentParser(description='Monitor metrics in a real-time time-series graph');
+        parser = argparse.ArgumentParser(description='Monitor metrics in a real-time time-series graph')
 
         group = parser.add_mutually_exclusive_group()
         group.add_argument('-m', '--monitor',
                             type = str,
-                            default = 'pcpu',
-                            help = 'Select from builtin monitors to feed data into monitor [default: pcpu]')
+                            default = None,
+                            help = 'Select from builtin monitors to feed data into graph [default: pcpu]')
         group.add_argument('-c', '--command',
                             type = str,
                             default = None,
-                            help = 'User command to feed data into monitor [example: ps -eo rss,pid,comm --no-headers]')
+                            help = 'User command to feed data into graph [example: ps -eo rss,pid,comm --no-headers]')
+        group.add_argument('-s', '--stream',
+                            type = str,
+                            default = None,
+                            help = 'Streaming user command to feed data into graph [example: tail -f file]')
+        group.add_argument('-f', '--file',
+                            nargs = '?',
+                            type = argparse.FileType("r"),
+                            default = None,
+                            const = sys.stdin,
+                            help = 'Continuously read file or STDIN to feed data into graph')
 
         parser.add_argument('-r', '--refreshrate',
                             type = int,
@@ -65,15 +78,32 @@ def initFromArgs(self, args: list[str] | None = None):
             self.list_monitors()
             sys.exit(0)
 
-        if args.command == None:
+        if args.monitor:
+            print("args.monitor")
+            self.monitor_type = MonitorType.BUILTIN
             monitor = os.path.join(os.path.dirname(__file__), 'monitors', args.monitor)
             if not os.path.exists(monitor):
                 print(f"No such builtin monitor `{args.monitor}`. Use `--list-monitors` to list available builtin monitors.")
                 sys.exit(1)
             self.monitor_cmd = monitor
-        else:
+        elif args.command:
+            print("args.command")
+            self.monitor_type = MonitorType.COMMAND
             self.monitor_cmd = args.command
-            self.monitor_type = MonitorType.USER
+        elif args.stream:
+            print("args.stream")
+            self.monitor_type = MonitorType.STREAM
+            self.monitor_cmd = args.stream
+        elif args.file:
+            print("args.file")
+            self.monitor_type = MonitorType.FILE
+            self.monitor_cmd = args.file
+        else:
+            # default to pcpu builtin monitor
+            print("default pcpu")
+            self.monitor_type = MonitorType.BUILTIN
+            pcpu = os.path.join(os.path.dirname(__file__), 'monitors', 'pcpu')
+            self.monitor_cmd = pcpu
 
         self.refresh_rate = args.refreshrate
         self.max_ticks = args.ticks
@@ -86,11 +116,11 @@ def initFromArgs(self, args: list[str] | None = None):
             self.tick_mod = 1
 
     def update_tick(self):
+        self.now = time.time()
         self.tick_index += 1;
         if self.tick_index == self.max_ticks:
             self.tick_index = 0
         self.tick_time = time.strftime("%H:%M:%S", time.localtime(time.time()))
-        self.mainwindow.rgn.setRegion((self.tick_index, self.tick_index))
 
     def list_monitors(self):
         print("Builtin monitors:")

src/grafmon/errordialog.py → src/grafmon/dialogs.py

@@ -9,3 +9,14 @@ def __init__(self, *args, **kwargs):
 
     def show(self):
         self.exec();
+
+
+class AlertDialog(QMessageBox):
+    def __init__(self, *args, **kwargs):
+        super(AlertDialog, self).__init__(*args, **kwargs)
+        self.setIcon(QMessageBox.Icon.Information)
+        self.setWindowTitle("Alert")
+        self.setStandardButtons(QMessageBox.StandardButton.Close)
+
+    def show(self):
+        self.exec();

src/grafmon/filemonitor.py

@@ -0,0 +1,68 @@
+from PyQt6.QtCore import (
+        QObject,
+        QThreadPool,
+        QRunnable,
+        pyqtSignal as Signal
+)
+
+from .context import Context
+from .dialogs import ErrorDialog, AlertDialog
+
+class FileMonitor:
+    def __init__(self, context: Context):
+        self.context = context
+        context.monitor = self
+        self.file = context.monitor_cmd
+        self.threadpool = QThreadPool()
+        self.reader = None
+
+    def update(self):
+        if self.reader == None:
+            self.reader = Reader(self.file)
+            self.reader.signal.data.connect(self.context.mainwindow.update_data)
+            self.reader.signal.error.connect(self.error)
+            self.reader.signal.done.connect(self.done)
+            self.threadpool.start(self.reader)
+
+    def error(self, exc, value):
+        print(f"Monitor error: {exc}: {value}", file=sys.stderr)
+        err = ErrorDialog()
+        err.setText(f"Monitor error: {exc}")
+        err.setInformativeText(value)
+        err.show()
+        self.context.app.exit(1)
+
+    def done(self):
+        if self.context.fatal_error:
+            return
+        self.context.timer.stop()
+        d = AlertDialog()
+        d.setText("File closed")
+        d.setInformativeText("The file has signaled that all data has been consumed.")
+        d.show()
+
+    def stop(self):
+        pass
+
+class ReaderSignals(QObject):
+    data = Signal(object)
+    error = Signal(object, object)
+    done = Signal()
+
+class Reader(QRunnable):
+    def __init__(self, *args, **kwargs):
+        super().__init__()
+        self.file = args[0]
+        self.signal = ReaderSignals()
+
+    def run(self):
+        file = self.file
+        try:
+            while line := file.readline():
+                data = line.strip().split(maxsplit=1)
+                self.signal.data.emit(data)
+        except:
+            exc, value = sys.exc_info()[:2]
+            self.signal.error.emit(exc, value)
+        finally:
+            self.signal.done.emit()