Create/publish the user documentation

[kelson42]

At this stage this seems necessary to publish a user documentation. Https://Readthedocs.com seems a good candidate.

[kelson42]

Thanks to @benoit74 for reporting the problem

[benoit74]

I'm not sure that we need to transform existing documentation.
Adding a paragraph on all scrapers README with a link pointing to existing one would be sufficient from my PoV.

[rgaudin]

Probably a good first step

[kelson42]

@benoit74 Your link is not the scraperlib documentation. Explaining the goals, the paradigm of the scraperlib and the dev facing API of the scraperlib is the puropose of this ticket.

[rgaudin]

I don't think this is the correct way to address what I believe the underlying issue is.
What we need is a general Make/Contribute-to/ a Scraper documentation. This would fit with the other documents referenced above and would be actually useful while a standalone scraperlib doc won't be of much help to do so.

Sure we could gen the auto-doc off the source code but the value vs just browsing the code on Github would be minimal. Just afraid this would close this ticket without helping onboarding.

[kelson42]

@rgaudin Not 100% sure I get you right. But I can only disagree with a comment which in a nutshell says "no need to document library API, devs who want to use it should read the code".

For the rest, general (cross repo) documentation can have a place in the "overview" repo. But, how to use the scraperlib does not match this definition IMO.

Lets discuss this in detail at the next meeting.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[josephlewis42]

As someone who just created a new scraper, here are my thoughts:

• It was difficult for me to understand what capabilities existed in the scraperlib versus what I was expected to build.
• It wasn't clear to me when the documents/code here, the GitHub wiki, or the OpenZim wiki was preferred.

I like automatically updating docs based on current source code e.g. GitHub pages triggered on release/merge because it's a faster way for me to understand the shape/intent of a library without diving into the details.

[rgaudin]

I like automatically updating docs based on current source code

We all do I believe and I think that scraperlib would be a good candidate! Do you want to take a look at it?
I think we mostly need this setup, with the appropriate actions and the pointers to the right docstring convention.

We'd host it at readthedocs I think, as we do for other projects.

I believe libzim uses sphinx but I have also used mkdocs in the past and I find it easy to configure, flexible, readable and good looking. It's an active and popular tool.

Here's a good example of config for it: https://github.com/pawamoy/aria2p/blob/main/mkdocs.yml

[josephlewis42]

@rgaudin sure, this is something I can look at probably in four-ish weeks assuming the issue is still open.

I've got a bit more to do with the DevDocs scraper to improve the UX after the release, once that's done I can scoot over to this assuming no other show-stoppers.