[Rivet] rivet-mkhtml recursively removes ${pwd}/plots without warning

[Holger Schulz]

Hi,

given that rivet 2.4 was released yesterday, I propose we collect
David's suggestions on a trello card for 2.5.

Cheers,
Holger


On 08/10/15 13:05, David Bjergaard wrote:
> Hi,
>
> Just because unix does something doesn't mean that makes it user friendly or
> sane. I haven't used rivet-mkhtml until today because I prefer dumping the *.dat
> files with rivet-cmphistos and then calling make-plots.  As a result, I had
> plots/dat and plots/pdf in ${PWD} and I would call rivet-cmphistos with -o
> plots/dat in order to avoid cluttering the ${PWD}.
>
> Given that /I/ made ${PWD}/plots and the files contained therein, I was
> surprised to find that the whole directory structure had been clobbered and
> replaced.
>
> When rivet replaces a file it created (.yoda) when I re-run the command, that's
> an expected side effect.  My objection is that command shouldn't be allowed to
> appropriate and replaces a directory that the user made for another purpose.
>
> David's suggestion that use put a hidden zero size file (whatever the python
> equivalent of "touch ${PWD}/plots/.mkhtml" is) and check for it would be the
> best compromise. That would keep the core behavior for people who expect their
> output to be over-written and it prevents people from clobbering plots that they
> might otherwise need.
>
> If you don't do anything, please at least change this:
> "You can overwrite an existing output directory."
> to
> "WARNING: This script automatically overwrites the existing output directory
> (default: ./plots)."
>
> That is a much more clear description of whats going to happen when reading the
> usage and options of the rivet-mkhtml script.
>
> Best,
>
>      David
>
>
> David Grellscheid<david.grellscheid at durham.ac.uk>  writes:
>
>> Hi!
>>
>> Please keep the core behaviour as it is now. Andy has mentioned most of
>> the arguments already. We've had it the other way round before, and that
>> was a real pain to work with. I much prefer the silent convenience of
>> overwriting the existing output.
>>
>> The issue is more in the naming I think. A directory name that has Rivet
>> explicitly in the name is less likely to have pre-existed. Or we check
>> for a hidden 0-size file in the directory before deleting, to make sure
>> the directory is one of ours.
>>
>>    David
>>
>>
>>
>> On 08/10/15 12:03, Andy Buckley wrote:
>>> Hi David,
>>>
>>> I'm not really sure why it would be a surprise. We equally don't warn or
>>> stop users from running rivet if the output .yoda file already exists,
>>> nor re. overwriting of the files and directories from rivet-cmp histos.
>>>
>>> The core Unix shell tools also don't complain when they would overwrite
>>> an existing file -- unless you specifically tell them to do so (for
>>> those which have such an option). Personally I prefer the convenience of
>>> the tools not objecting to overwrites. A priori if I run any command
>>> that would logically write into the same dir as a previous run, I should
>>> not expect my existing data to remain intact -- that's not just
>>> restricted to Rivet&  friends!
>>>
>>> I was going to add the printed warning that you suggested, then changed
>>> my mind -- by the time they've seen the message the damage is already
>>> done, and we'd "need" to add similar messages to every tool in the Rivet
>>> (and YODA) suite. It just seems like extra noise to me. But maybe others
>>> disagree?
>>>
>>> Andy
>>>
>>>
>>> On 08/10/15 11:48, David Bjergaard wrote:
>>>> Hi All,
>>>>
>>>> Luckily I didn't lose any plots that I was attached to, but I did have a ./plots
>>>> folder that was using to store intermediate results.  Removing any files without
>>>> warning the user is kind of scary which is what I think this line does:
>>>>       >  if os.path.exists(opts.OUTPUTDIR) and not os.path.realpath(opts.OUTPUTDIR)==os.getcwd():
>>>>       >      import shutil
>>>>       >      shutil.rmtree(opts.OUTPUTDIR)
>>>> Why can't you throw an exception and say "OUTPUTDIR not empty, please use -f to
>>>> force overwriting", or less ideal but still better "OUTPUTDIR is not empty,
>>>> removing" so that its less of a surprise when all the subdirectories in
>>>> OUTPUTDIR have mysteriously vanished.
>>>>
>>>>       David
>>>> _______________________________________________
>>>> Rivet mailing list
>>>> Rivet at projects.hepforge.org
>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>
>>>
> _______________________________________________
> Rivet mailing list
> Rivet at projects.hepforge.org
> https://www.hepforge.org/lists/listinfo/rivet