Cache data elsewhere / account for merged data

[dhcole]

Current behavior is to optionally store all of the data for a particular remote resource in the _data directory. This is done after merging with existing local data.

@mbland raised an issue about whether this should be more configurable.

@mbland: So there is one bit of danger here. If target contains any non-empty Array members that are duplicated in source, those duplicates will appear in target. Since the original intent of this plugin was merging data from Midas with data stored in the Hub, this makes sense. But keeping a full cached copy of the data in the repo is a different use case.

Hence, I'm wondering if the cached version of the data should live somewhere like _data/cached. Then, in this plugin, in the !config case, we can use: HashJoiner.promote_data site.data, 'cached'.

@dhcole: _data/cached is non-standard though. I'd rather not introduce new conventions. We can keep this simple. The data that this would cache accounts for merging remote data with local data (but only if you choose to cache).

Since caching is optional to begin with, I think this behavior is fine.

@mbland: d['cache'] could be the path to the cached version of the data. I don't see anything wrong with adding new conventions if they solve real problems.

Moved here from https://github.com/18F/18f.gsa.gov/pull/627/files#r26708063 so we can have a discreet conversation about this :)

[mbland]

What I didn't mention in that other thread is that I noticed this problem during my first experiments embedding this plugin in the Hub. :-) The Hub currently keeps a "public" cache of _data/private stored directly in _data. This means that when I use Jekyll_Get, I get two copies of my skills, etc.

Unrelated, but mentioning while I'm thinking about it: When it comes to types besides Array or Hash, HashJoiner.deep_merge will overwrite values in target with values from source. I believe this is as you'd expect, but worth making explicit.