Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REST API certificate based authentication #534

Open
Gunni opened this issue Apr 24, 2019 · 27 comments · Fixed by #637
Open

REST API certificate based authentication #534

Gunni opened this issue Apr 24, 2019 · 27 comments · Fixed by #637

Comments

@Gunni
Copy link

Gunni commented Apr 24, 2019

Since mntner objects can be locked using key-cert objects, and Client Certificate authentication is a thing.

I hereby propose its addition to the RIPE REST API.

In addition to being able to create, update, and delete objects when authenticated you could also allow requesting the full/unfiltered/unmodified object owned by the authenticated mntner.

@eshryane eshryane self-assigned this Apr 29, 2019
@eshryane
Copy link
Member

Hello @Gunni thanks for your suggestion, I'll add this to our product backlog for further investigation.

The RIPE REST API currently only supports passwords and/or SSO credentials. This sounds like a good alternative. How likely is this to be adopted by users?

There is some related discussion around authentication in the NWI-8 proposal on the DB-WG.

@eshryane
Copy link
Member

eshryane commented May 17, 2019

Hello @Gunni, it appears that support for PGP keys was dropped from Client Certificate authentication in TLS 1.3.

"The OpenPGP certificate type [RFC6091] MUST NOT be used with TLS 1.3."

https://crypto.stackexchange.com/questions/63981/why-did-tls-1-3-prohibit-pgp-authentication
https://tools.ietf.org/html/rfc8446#section-4.4.2

Does it still make sense to consider this for Whois?

@eshryane
Copy link
Member

@Gunni Thanks for clarifying! Indeed we already support X509 key-certs, we will investigate this further.

@Gunni
Copy link
Author

Gunni commented Feb 21, 2020

Has there been any luck with this @eshryane? x509 cert auth is very well defined and most clients I know of support it. And is infinately better than having to add a md5 password to the mntner.

@eshryane
Copy link
Member

Hi @Gunni No update, but I haven't forgotten (we were discussing it last week in fact).

There's a number of areas we'll need to investigate in order to implement this:

  • Configure the Apache httpd proxy to pass the the X509 certificate in a header

    • We don't want to create a dependency on this solution however, in case it makes it difficult to migrate away from httpd later
    • We also don't want to allow weak keys (short length or insecure algorithm) to be allowed (as it would bypass the existing SSL restrictions).
    • We need to do this without disturbing the existing use of SSL.
    • We need to test this fully, and document supported use case(s), to allow our users to actually implement it on the client side.
  • Configure Whois to make use of the X509 certificate header

    • This is relatively straightforward, we can create a new credential type, and look for a matching X509 key-cert on a maintainer when doing the authentication.

It's not a huge amount of work in total, but enough that it hasn't been a priority. I agree that it's long overdue to replace MD5 (not sure what the migration path will look like however, there probably won't be huge take-up of client certificate authentication).

Now that you've reminded me, I will setup a test in our Release Candidate environment.

@eshryane
Copy link
Member

@Gunni I've completed the httpd proxy configuration to forward a client certificate to the backend Whois server (if provided). We'll work on the necessary changes to Whois next, and deploy to RC in the next few weeks for testing.

@eshryane eshryane modified the milestones: whois-1.93, whois-1.97 Feb 25, 2020
@eshryane
Copy link
Member

@Gunni We didn't have time to implement the necessary Whois changes before the 1.97 release, and we are waiting for a security review of our implementation, but I'm still planning to deploy this before the next RIPE meeting.

@eshryane eshryane modified the milestones: whois-1.97, whois-1.97.1 Mar 18, 2020
@eshryane
Copy link
Member

eshryane commented May 8, 2020

@Gunni We didn't have time to add this feature to Whois 1.97.1, but we still plan to implement it. I will have an update for the DB-WG session at RIPE80 next week.

@eshryane eshryane modified the milestones: whois-1.97.1, whois-1.97.2 May 8, 2020
@eshryane eshryane linked a pull request Jun 22, 2020 that will close this issue
@eshryane
Copy link
Member

@Gunni We've merged initial support for this feature.

We will include the change in the Whois 1.97.2 release, but it will be turned off in production for now, until the community have tested it, and we also plan a security review.

@daniel-brenot
Copy link

Has this been enabled for production yet or is it still for development only?

@eshryane
Copy link
Member

Hi @daniel-brenot @Gunni We have implemented this feature, but we haven't enabled in production yet.

We are planning to do a security review, and we are now also replacing our httpd proxy layer with F5, which we will depend on for SSL termination and forwarding the client certificate information. We hope to enable this soon.

@Gunni
Copy link
Author

Gunni commented Nov 22, 2021

We hope to enable this soon.

What about enabling it on test? Are we able to test it there? Do you have a link to docs for test db rest api?

@eshryane
Copy link
Member

Hi @Gunni We plan to initially enable it in the Release Candidate environment, but we're going to wait until we switch the proxy layer from httpd to F5, hopefully we'll do this by the end of the year.

Once the community have tried it out, and after it's passed a security review, we plan to enable it in production.

@eshryane
Copy link
Member

eshryane commented Nov 22, 2021

Hi @Gunni I went ahead and enabled client certificate authentication in Release Candidate, so feel free to test:
https://www.ripe.net/manage-ips-and-asns/db/release-notes/rc-release-candidate-environment

You'll need to generate a key-pair, then create a keycert object with the public key and add that to a maintainer. Then you should be able to connect over TLS with your key-pair, and authenticate a Whois update as the maintainer.

Client certificate authentication should work on rest-rc.db.ripe.net and syncupdates-rc.db.ripe.net.

RC is still using httpd to proxy requests, but we are investigating client certificate support in F5 now.

@eshryane
Copy link
Member

The whois code changes are in #637

@Gunni
Copy link
Author

Gunni commented Nov 22, 2021

I have verified that CREATE, UPDATE, DELETE objects works with client cert auth.

Nice!

However I still get Filtered objects back when READing, even if I am authenticated while querying.

    }, {
      "name" : "auth",
      "value" : "SSO",
      "comment" : "Filtered"
    }, {

Please ensure getting unfiltered objects works when requests are authenticated!

Edit: Additionally, when doing updates like this, the update emails that normally say: Changed by SSO account: [email protected] has nothing about who updated the object.

@eshryane
Copy link
Member

Hi @Gunni Thanks for testing! Good to hear it's working for you.

We haven't implemented client certificate authentication for reads yet (GET requests), only for updates (POST, PUT, DELETE). We will add this in the next release.

@eshryane
Copy link
Member

Hi @Gunni this feature is still on our list, we had some problems making it optional with the F5 loadbalancer we're currently using. We plan now to move HTTPS support to the backend application and allow client certificate authentication there.

@Gunni
Copy link
Author

Gunni commented Dec 30, 2023

Hey, it's been a year, are there any news?

@eshryane
Copy link
Member

eshryane commented Jan 2, 2024

Hi @Gunni Happy New Year. Yes I presented on client certificate authentication at RIPE87 in Rome:
https://ripe87.ripe.net/programme/meeting-plan/db-wg/

We've implemented Client Certificate authentication together with backend HTTPS in the Whois 1.109 release.

Unfortunately there is a blocking issue that's preventing us from enabling it just yet: the client certificate request in the TLS handshake causes browsers to ask the user to choose a certificate, which annoys users who don't want to use this feature. This happens not only when clicking on Whois REST API links but also for requests within single page web applications.

We're going to avoid this issue by creating a separate fully-qualified domain name just for client certificate authentication, and deploy in the next Whois release.

Apologies for the delay.

@eshryane
Copy link
Member

Hi @Gunni I've documented how to use Client Certificate Authentication in the DB documentation with an example:
https://apps.db.ripe.net/docs/Appendices/Appendix-I--Client-Certificate-Authentication/

Use the host name rest-cert.db.ripe.net instead of rest.db.ripe.net, we separated the service as a popup will appear in browsers asking users to choose a certificate when following (GET) REST API links, which is annoying.

@eshryane
Copy link
Member

I'm closing the issue as client certificate authentication is now implemented and deployed in Whois.

@eshryane
Copy link
Member

eshryane commented May 16, 2024

Reopening issue as @Gunni reported:

"But it still does not provide unfiltered fields for authenticated requests, which is what I need for to implement idempotency in clients that maintain records, I cannot see what auth sso are on a mntner so i would need to post the object every time to ensure it is as it should be."

Apologies I've confirmed this is not working, we'll fix in the next Whois release.

@eshryane eshryane reopened this May 16, 2024
@MiguelAHM
Copy link
Contributor

Hi @Gunni
I hope you are doing well.

I am pleased to inform you that Client Certificate Authentication for lookups is now supported. With this update, you can retrieve unfiltered information for your objects using your certificate.

This feature is included in the 1.113 release.

Could you please confirm if this resolves the issue you were experiencing?

@Gunni
Copy link
Author

Gunni commented Jul 1, 2024

It partially works!

I made my key-cert using this (RSA is so 1977)
openssl req -x509 -newkey ed25519 -keyout RIPE.key -out RIPE.crt -sha512 -days 365 -nodes -subj '/C=IS/CN=...'

Then used this query:
curl -v --header "Content-type: application/json" --key RIPE.key --cert RIPE.crt https://rest-cert.db.ripe.net/ripe/mntner/MNT-GUNNAR.json

What works is that i get the auth: sso emails, fantastic! However, other fields are missing/filtered, such as upd-to, mnt-nfy, and notify.

In addition to that, the last tag still says Filtered

{
      "name" : "source",
      "value" : "RIPE",
      "comment" : "Filtered"
}

It should not be filtered at all if I am authenticated for that object, so I'll give partially done.

Think of it this way, I should be able to generate an object on my side and diff every field, including those above, and remarks, and if not correct, I'd submit a PUT for it.

Notes

Things that I noticed while testing this

Documentation issue

Noticed while making my key, documentation of key-cert is outdated. Only specifies PGP keys. While online documentation is fine.

$ whois -h whois.ripe.net -- -v key-cert
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See https://apps.db.ripe.net/docs/HTML-Terms-And-Conditions

The key-cert class:

      A key-cert object is a database public key certificate
      that is stored on the server and may be used with a mntner
      object for authentication when performing updates.
      Currently only PGP/GnuPG keys are supported.

key-cert:       [mandatory]  [single]     [primary/lookup key]
...

The content of the attributes of the key-cert class are defined below:

key-cert

   Defines the public key stored in the database.

     PGPKEY-<id>

     <id> is  the PGP key ID of the public key in 8-digit
     hexadecimal format without "0x" prefix.

method

   Defines the type of the public key.

     Currently, only PGP keys are supported.

Also the format of the key-cert field is not documented anywhere I could find. Turns out it must be AUTO-nnn.

Email notification

The email notification i get when someone updates a record that I watch using mnt-nfy does not contain what SSO account or x509 cert was used to perform the action.

p.s. Short OpenPGP key IDs are insecure.
p.p.s. When this is done, do we get a News post for this feature? 😁

@eshryane
Copy link
Member

eshryane commented Jul 1, 2024

Hi @Gunni thanks for your feedback!

REST API lookups are filtered by default, even if you're authenticated. You need to add the unfiltered flag to REST API lookups to request the entire object including attributes which may contain personal data. Authenticated lookups with a password work the same way. I do think it's confusing though, we will review this behaviour.

Thanks for pointing out the documentation issue, we will fix it.

I agree that short key-ids are insecure, but we are using it as a primary key, not for authentication, and it must be unique. I think using the full key fingerprint as the primary key would be unwieldy. We are open to feedback to change this though (just how we currently do it).

We do plan to deprecate MD5 hashes as announced at RIPE88, client certificate authentication is a good alternative, and this will be announced as part of the impact analysis.

Thanks again for testing our implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants