Interested in contributing

[ajberkley]

The lisp-stat.dev page says this is used regularly by some people. If so, I'm thinking of contributing some. I do a lot of statistical analysis in Common Lisp using a large internal collection of tools at my day job, so I have a lot of experience (and bias) with statistics and interactive and non-interactive workflows using Common Lisp plus Matlab. I'm mostly interested in interactive work where one is exploring, analyzing, and fitting data (in particularly the combination of a web browser plus repl in emacs or repl in web-browser). In particular I would like to go beyond the emacs repl + static plots workflow (which is typically what I have been using). I think with some work, https://github.com/ajberkley/plotly-user (which is CLOG https://github.com/rabbibotton/clog based, in combination with some support for d3 directly) could be straightforwardly extended to provide a full interactive "studio" like environment. Things like like graphs with editable annotations, notes pages, interactive code snippets, with the full workspace saveable and loadable is important. That's the direction I'd want to work on. While it sounds on the surface like a huge piece of work, the heavy lifting has already been done by CLOG, plotly, and d3.

I think I'm writing this note to see how active this project is and whether people are still excited about it.

[snunez1]

Well, at least one of us will be. :-)

This community tends, like most CL users, to be rather quiet until/unless something goes wrong and the past contributors have more or less just added their features and made a pull request.

There's a few ways that might be worth exploring to shape an EDA style workflow, some further developed than others. Here's a brief summary of where things are at:

• The first may be an easy addition to what you currently do: emacs-vega-view, which can render plots directly in emacs. I think this is a better way to render visualisations than the browser. The current method of writing out HTML and calling the system browser has always been somewhat fragile.
• Mikel Evins has been working on an integration with a web view component, (clio), that I think would provide a nice interactive environment and allow us to create a hybrid web/repl app. Mikel has experience in designing this sort of thing and I think he's on the right track, but Mikel gets easily distracted (Hi Mikel! :-)) and whilst close, clio isn't quite there. Perhaps another person with a request will be enough to push this over the line. As part of this there's a data server layer to expose data frames via http and render in a nice table via datatables, though it could use other table renderers.
• It is also possible to use Python in RStudio, and RStudio is a nice IDE for EDA. I've often thought that this integration would really give us a leg up in EDA environments. A brief look suggests it's not that difficult, but would take some rummaging around in RStudio.
• Finally there's some work I'm currently doing on creating common-lisp-jupyter and lisp-stat features to be added to devcontainers, and then using notebooks in VS Code. I've run notebooks in VS Code and whilst I don't really like notebooks for EDA, many people do and this might be the shortest path from A to B. The Alive extension isn't quite my cup of tea, but it looks like SLIMA could be ported without too much work, certainly less work than developing an entirely new GUI. Neil is quite helpful in answering questions about the integration.

That's where things stand now. I don't want to discourage development of a CLOG GUI for Lisp Stat. It's been suggested before and isn't a bad idea. Perhaps some of these other efforts are a better fit or less effort for what you want.

Oh, you do know that you can currently do EDA in notebooks? I assumed in this discussion that you wanted something other than notebook style EDA. See the example notebooks for an EDA style analysis in Jupyter Lab.

[ajberkley]

I will experiment with jupyter lab plus lisp-stat -- if it does everything I want, no need for me to reimplement the wheel.

[snunez1]

Just read plotly-user README on github. You'll want to investigate clio for sure. Mikel is all about interactivity in GUIs and that's what CLIO is designed for. I pointed Mikel to this discussion; if he doesn't show up here you might want to comment on the clio repo. Interacting with data in a browser is the main goal of clio/lisp-stat.

[mikelevins]

Clio's not really a substitute for jupyter notebooks. It's more like a toolkit for building things like jupyter notebooks (and other kinds of UIs). Moreover, it's very much a work in progress, and probably not yet useful for most people. Progress proceeds in fits and starts as I find time to work on it.

I started working on it after building a number of browser-based UIs for Lisp programs that I built at work. I work for an engineering R&D company building experiments and proofs of concept, and it's pretty common for us to build browser-based UIs for our projects. As you might imagine, I've done essentially the same work over and over on UIs like this, and I wanted to pull together a toolkit I could use to avoid doing the same work over and over.

I'm also influenced by a history of working on and with a lot of different UI frameworks on a lot of different platforms. That history has cultivated some preferences in how I want things to work, and Clio reflects them.

If you want to do something with Clio, I'll be happy to help, but be aware that in its current state it's essentially a collection of parts, rather than a polished framework. Those parts are already helpful in my day-to-day work, but might not be useful to much of anyone else, unless they happen to be doing the same kinds of things over and over that I do.

Nevertheless, if you want to tinker with Clio, I'll be happy to help.

It's probably also useful to know that clio comprises three subprojects:

• backend is the Lisp HTTP server and event-handling code
• stage is a Tauri-based windowing application skeleton that I'm preparing. When finished, it will provide a convenient way to package Clio apps as standalone applications.
• appexec is some boilerplate for building a command-line executable from backend, for use with stage or for cases where you want the browser-based UI but not the standalone windowing application (for cases, for example, when you want to deliver your UI as a web application rather than as a locally-launched windowing app)

The repo is here: https://github.com/mikelevins/clio