Show files under hidden directories in job composer

[Bubballoo3]

Fixes #2268 by rejecting only hidden files, not files under hidden directories. Modifies and expands testing of this method

[Bubballoo3]

I am having some trouble developing the myjobs app, so these changes are not tested at the moment. It does seem to be passing all of the automated testing, but not sure how extensive this is for the myjobs app.

[johrstrom]

Were you able to boot up the job composer to test these changes?

[Bubballoo3]

I was able to boot the job composer, but this seems like a difficult problem to test. Even if we put the scripts for the job in a hidden folder, myjobs moves them to a designated project folder that will never be hidden. The original discourse worked out of .ondemand as the root folder, and so set the OOD_DATAROOT variable to #{HOME}/.ondemand/.../.../myjobs. I can't seem to get the OOD_DATAROOT variable to change (tried .env.local and .env.override) and when I try in the dev container I can't get the job composer working. I also suspect it would be a good deal of work to configure the dev container to read out of a .ondemand folder. Any suggestions?

[johrstrom]

Setting OOD_DATAROOT in .env.local should work, though you'd need to bounce your PUN for it to get picked up. I can take a look at this a bit more later this week.

[johrstrom]

Tested this out and it works. I think we're good, though I think I'd like to use @path.basename just for simplicity over split('/')[-1].

[johrstrom]

Thanks! Works well for me.