Request - private curatorial notations for records

[mkoo]

This issue originated in #3536 and became stale-- after discussions, here's the refreshed summary:

Need: a means to allow curatorial staff to leave private (non-public) record level notes about condition or preservation need especially in the art collections.

Options considered included:

1. two new encumbrances: mask part attribute condition report and mask part attribute preservation need
2. new private record attributes
3. new curatorial remarks that are always private

Main barrier/consideration is resource costs-- encumbrances are 'expensive' and slows exports down but they are essential. However if we can avoid unnecessary encumbrances then we all benefit.

Last time the AWG discussed this, we were favoring option 3 (if I'm not mistaken!)

[happiah-madson]

Strong preference for #3.

[campmlc]

I support option 3.

[dustymc]

I made ArctosDB/dev#118 for this, but I'm now questioning that.

encumbrances are 'expensive' and slows exports down

Clarification: As currently implemented encumbrances slow everything down. ArctosDB/dev#118 will still have computational costs, but they should be much more manageable (and understandable) under such a system.

I was also going to suggest we develop scalable policy before this is implemented. If this works like I think it should, Phase Two will involve record attributes, of which there are very many. The "full" implementation of this would simply double that - we'd have a public 'tail condition' and a private 'tail condition' (the existence of which suggests we're already long-overdue for more-developed policies regarding what is an appropriate attribute), which seems neither sustainable nor usable.

Now I'm wondering if we can actually implement this entirely against record attributes, or if parts (and presumably other kinds) are necessary? The code isn't written and I'm guessing to some extent, but running 'private types' against one table is probably well under half the computational expense of two, but is also simpler to use and manage (restricted data go in one place, easy-peasey, nowhere to get lost) and possibly sufficient to cover the critical use cases. Would having even part-related things under one (or a very few) private record attribute(s) be sufficient, or is attaching that information to specific parts truly critical?

Adding to AWG agenda for further clarification, "as simple as possible, as complex as necessary" seems particularly relevant here.

[happiah-madson]

At the risk of opening cans of worms, has there been discussion about both record level and part level curatorial remarks? I've just been speaking with @dandistel who is contemplating encumbering all remarks because some of them are more curatorial than public.

[dustymc]

both record level and part level curatorial remarks

So two (three[four]) things:

1. That's why we're here, do we REALLY need both or can record-level sufficiently cover the bases (and of course more resources would change everything, but as of now any sort of simplification or conservation seems to me like it should carry a great deal of weight).
2. What's been proposed would involve attribute type (because I think that's probably the simplest realistic solution, PLEASE let me know if there's an easier option - eg would cataloged_item.private_remark and specimen_part.private_remark do whats needed?!? I'd be very happy if so...), which sets the stage for more than remarks (see aforementioned 'private version of thing which probably shouldn't exist' attribute type).
3. (Some guidance from The Community would be very useful. I can only imagine ever using this for something that my local counsel think would stop a competent lawyer waving a FOIA request - otherwise it's probably the public's information, they paid for it, why make it difficult for them to access it?? Surely some hippiness shining through but Community guidance still seems really useful in shaping both use and development.)
4. ([Arctos has no security team. I'll do my best to protect anything behind an encumbrance, whatever shape it might take, but that's not exactly a substitute for the normal 'expensive people with expensive tools and no other job' - who still failed right before you got those dozen or so 'sorry, your information has been compromised....' notices you've probably received over the years. Possibly those things which are truly private need something more like https://github.com/ArctosDB/internal/issues/357 and we shouldn't be considering this at all - IDK, guidance still seems useful.])

[happiah-madson]

That's why we're here, do we REALLY need both or can record-level sufficiently cover the bases

I think my comment about both is that when migrating, we had specimen (record) and DNA or sample (part) remarks that are things like "oh, we found this in a not so great place, salvaged it, and now it's over here" or "I assume the elution volume was 200 uL because I forgot to write it down in my lab notebook so here's hoping." Because these remarks were never public in our last database, they are written super colloquially and perhaps would have been written differently if they were public.

eg would cataloged_item.private_remark and specimen_part.private_remark do whats needed

From my view, yes, this would do what is needed.

otherwise it's probably the public's information, they paid for it, why make it difficult for them to access it??

For us, it's probably just tone. It's editorial rather than factual information that migrated over in remarks.

Arctos has no security team

Understood!

[campmlc]

I think this would work for us as well: cataloged_item.private_remark and specimen_part.private_remark
We would like to be able to record a private part remark such as "oops, this vial of tissue was discovered under an ultralow freezer and obviously fell out at some point for some extended period of time and is now very rotten" at the same time as saying "condition = poor" in the public part remarks. I can also see use for private remarks at the cataloged item level, e.g this specimen had its tail pulled off by a group of kindergardeners on a school tour and it's been glued back on - we need to restrict future handling etc. I assume the art and cultural collections have even more issues with needing private remarks fields.

[dustymc]

this would work for us as well: cataloged_item.private_remark and specimen_part.private_remark

Please confirm that you mean as a way to handle https://arctos.database.museum/info/ctDocumentation.cfm?table=ctencumbrance_action and an alternative to ArctosDB/dev#118.

[campmlc]

If I understand correctly, we would not be using encumbrances for this, but rather using public/private flags or specifically designated private remarks fields for part and record attributes. This is what I thought I was supporting - option 3 above.

[happiah-madson]

Please confirm

I do. But I do think the fields could be more general, ie:

cataloged_item.private_remark and cataloged_item_part.private_remark

or

record.private_remark and record_part.private_remark

[dustymc]

not be using encumbrances for this

I'm trying to find a sustainable replacement for encumbrances (possibly one which does some things have have been requested for years - like allowing data to be born encumbered - and maybe simplifies some complexity, behind which people clearly get lost from time to time). ArctosDB/dev#118 is a move, not an addition. Introducing some even-simpler always-restricted field would only be relevant here if it's a replacement for some existing encumbrance.

(Bigger-picture, I'm struggling with the usual: Are things they way they are because users were trying to shoehorn various data into the structure as they understood it? - in which case a new always-private field or three might be just the ticket - or does the existing structure do precisely what needs to be done? - in which case ArctosDB/dev#118 is probably the correct first step towards a model we can afford.)

record.private_remark and record_part.private_remark

That's table.field terminology but those aren't tables so ???? (I think we're on the same page anyway!)

[happiah-madson]

that's table.field terminology but those aren't tables so ???? (I think we're on the same page anyway!)

😆😆😆😆😆😆😆😆😆😆😆😆 whooooopseeeeeeeeeee (I agree that we are on the same page)