[Rivet] rivet-mkhtml recursively removes ${pwd}/plots without warning

[David Bjergaard]

That sounds reasonable to me. Here's a patch for your troubles.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: rivet-mkhtml.patch
Type: text/x-diff
Size: 738 bytes
Desc: not available
URL: <https://www.hepforge.org/lists-archive/rivet/attachments/20151008/598b6362/attachment.patch>
-------------- next part --------------


Andy Buckley <andy.buckley at cern.ch> writes:

> We could alternatively change the rivet-mkhtml behaviour to write out
> to a rivet-plots directory. Then the name is less generic, and we
> don't have to add extra logic and "magic" like hidden files...
>
> Andy
>
>
> On 08/10/15 13:05, David Bjergaard wrote:
>> Hi,
>>
>> Just because unix does something doesn't mean that makes it user friendly or
>> sane. I haven't used rivet-mkhtml until today because I prefer dumping the *.dat
>> files with rivet-cmphistos and then calling make-plots.  As a result, I had
>> plots/dat and plots/pdf in ${PWD} and I would call rivet-cmphistos with -o
>> plots/dat in order to avoid cluttering the ${PWD}.
>>
>> Given that /I/ made ${PWD}/plots and the files contained therein, I was
>> surprised to find that the whole directory structure had been clobbered and
>> replaced.
>>
>> When rivet replaces a file it created (.yoda) when I re-run the command, that's
>> an expected side effect.  My objection is that command shouldn't be allowed to
>> appropriate and replaces a directory that the user made for another purpose.
>>
>> David's suggestion that use put a hidden zero size file (whatever the python
>> equivalent of "touch ${PWD}/plots/.mkhtml" is) and check for it would be the
>> best compromise. That would keep the core behavior for people who expect their
>> output to be over-written and it prevents people from clobbering plots that they
>> might otherwise need.
>>
>> If you don't do anything, please at least change this:
>> "You can overwrite an existing output directory."
>> to
>> "WARNING: This script automatically overwrites the existing output directory
>> (default: ./plots)."
>>
>> That is a much more clear description of whats going to happen when reading the
>> usage and options of the rivet-mkhtml script.
>>
>> Best,
>>
>>      David
>>
>>
>> David Grellscheid <david.grellscheid at durham.ac.uk> writes:
>>
>>> Hi!
>>>
>>> Please keep the core behaviour as it is now. Andy has mentioned most of
>>> the arguments already. We've had it the other way round before, and that
>>> was a real pain to work with. I much prefer the silent convenience of
>>> overwriting the existing output.
>>>
>>> The issue is more in the naming I think. A directory name that has Rivet
>>> explicitly in the name is less likely to have pre-existed. Or we check
>>> for a hidden 0-size file in the directory before deleting, to make sure
>>> the directory is one of ours.
>>>
>>>    David
>>>
>>>
>>>
>>> On 08/10/15 12:03, Andy Buckley wrote:
>>>> Hi David,
>>>>
>>>> I'm not really sure why it would be a surprise. We equally don't warn or
>>>> stop users from running rivet if the output .yoda file already exists,
>>>> nor re. overwriting of the files and directories from rivet-cmp histos.
>>>>
>>>> The core Unix shell tools also don't complain when they would overwrite
>>>> an existing file -- unless you specifically tell them to do so (for
>>>> those which have such an option). Personally I prefer the convenience of
>>>> the tools not objecting to overwrites. A priori if I run any command
>>>> that would logically write into the same dir as a previous run, I should
>>>> not expect my existing data to remain intact -- that's not just
>>>> restricted to Rivet & friends!
>>>>
>>>> I was going to add the printed warning that you suggested, then changed
>>>> my mind -- by the time they've seen the message the damage is already
>>>> done, and we'd "need" to add similar messages to every tool in the Rivet
>>>> (and YODA) suite. It just seems like extra noise to me. But maybe others
>>>> disagree?
>>>>
>>>> Andy
>>>>
>>>>
>>>> On 08/10/15 11:48, David Bjergaard wrote:
>>>>> Hi All,
>>>>>
>>>>> Luckily I didn't lose any plots that I was attached to, but I did have a ./plots
>>>>> folder that was using to store intermediate results.  Removing any files without
>>>>> warning the user is kind of scary which is what I think this line does:
>>>>>       > if os.path.exists(opts.OUTPUTDIR) and not os.path.realpath(opts.OUTPUTDIR)==os.getcwd():
>>>>>       >     import shutil
>>>>>       >     shutil.rmtree(opts.OUTPUTDIR)
>>>>> Why can't you throw an exception and say "OUTPUTDIR not empty, please use -f to
>>>>> force overwriting", or less ideal but still better "OUTPUTDIR is not empty,
>>>>> removing" so that its less of a surprise when all the subdirectories in
>>>>> OUTPUTDIR have mysteriously vanished.
>>>>>
>>>>>       David
>>>>> _______________________________________________
>>>>> Rivet mailing list
>>>>> Rivet at projects.hepforge.org
>>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>>
>>>>
>>>>