Generating machine-readable results

[mrshirts]

One nice suggestion was to generate machine-readable results for the output of physical_validation. The steps would be

1. Determine a JSON schema for the results
2. The functions fill in the schema.
3. the logfile is actually the output of running the JSON internal representation through an interpreter.

This is lower priority, but a good idea overall - would make it easier to have GROMACS take return values, or to have other programs incorporate results in an automated way.

[mikemhenry]

I think it would also be useful to access these values in python, so something like this could be used:

validation = pv.kinetic_energy.distribution(res_low, verbosity=2, strict=True,
                               filename=sysplot + '_low_mb')
if validation.Kolmogorov_Smirnov_test < .8:
    raise Exception("Null Hypothesis Rejected!")

This is just an example, but it would be nice for post processing scripts to be able to work with things in python directly, instead of having physical_validation write out JSON, then just read JSON back in. Doing the python API first might help inform the JSON schema as well.

[mrshirts]

Right, so one could imagine writing it to a dictionary that can be queried, writing the dictionary to a JSON, and passing the dictionary to a separate function that spit out human readable information (could play with this a little bit so you got information written to stdout during the process, not just after, if desired).

[mrshirts]

Any thoughts on roadblocks to getting this implemented, @ptmerz? I think this is one thing that can assist on making this easier to put in workflows.

[ptmerz]

I think it would be very helpful to have a concrete use case, and develop an implementation fulfilling the requirements of this use case (while remaining as general as possible, of course!). Bonus points for two or more use cases, to avoid losing generality :)

[ptmerz]

We don't seem to have a concrete use case, so this is on hold for now.

[ptmerz]

For example, the return value of the integrator test is not convincing. Could we instead return a list of time steps and rmsds? This would e.g. allow to plot a comparison of several tests.