Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intel AMT certificate provisioned with wrong common name FQDN | only works by IP #6592

Open
k1n6b0b opened this issue Dec 7, 2024 · 16 comments
Labels

Comments

@k1n6b0b
Copy link

k1n6b0b commented Dec 7, 2024

Describe the bug
The certificate loaded during provisioning does not use the dedicated FQDN specified in AMT config, im not sure how to specify it (or reprovision?)

To Reproduce
Steps to reproduce the behavior:

Configure AMT to use a dedicated IP and dedicated FQDN
Provision with MeshCentral

curl https://10.0.0.124:16993 --insecure -v #works

curl https://IntelAMT-6398b0b2a9ff:16993 --insecure -v --resolve IntelAMT-6398b0b2a9ff:16993:10.0.0.124 #works

However
curl https://dedicated-fqdn.xxx.yyy --insecure -v
fails because the certificate doesnt match

Certificate:
image

AMT config
image

Expected behavior
Maybe i did something out of order, but i'd expect the certificate loaded by MeshCentral to match the Computer Host Name . Domain Name

Screenshots
See above

Server Software (please complete the following information):

  • OS: Debian 12
  • Virtualization: Proxmox LXC
  • Network: Local certificate by certbot (symlink'd)
  • Version: v1.1.33
  • Node: v20.18.1

Client Device (please complete the following information):

  • Device: Intel BNUC11TNHv70L00
  • OS: Proxmox
  • Network: Local
  • Browser: Google Chrome

Additional context
Add any other context about the problem here.

Your config.json file

{
  "$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json",
  "__comment1__": "This is a simple configuration file, all values and sections that start with underscore (_) are ignored. Edit a section and remove th>
  "__comment2__": "See node_modules/meshcentral/sample-config-advanced.json for a more advanced example.",
  "settings": {
    "cert": "meshcentral.xxx.yyy",
    "_WANonly": true,
    "_LANonly": true,
    "_sessionKey": "MyReallySecretPassword1",
    "_redirAliasPort": 80,
    "_port": 80,
    "_redirPort": 81,
    "_aliasPort": 443,
    "_TLSOffload": false
  },
  "domains": {
    "": {
      "title": "XXX",
      "title2": "meshcentral",
      "_minify": true,
      "_newAccounts": true,
      "_userNameIsEmail": true,
      "agentConfig": [
        "webSocketMaskOverride=1"
      ],
      "certUrl": "https://meshcentral.xxx.yyy:443/"
    }
  },
  "_letsencrypt": {
    "__comment__": "Requires NodeJS 8.x or better, Go to https://letsdebug.net/ first before trying Let's Encrypt.",
    "email": "[email protected]",
    "names": "myserver.mydomain.com",
    "skipChallengeVerification": true,
    "production": false
  }
}

@k1n6b0b k1n6b0b added the bug label Dec 7, 2024
@si458
Copy link
Collaborator

si458 commented Dec 7, 2024

this is not a bug,
the AMT on the remote device will have its own self-signed intel AMT certificate https://127.0.0.1:16993
and the same goes for MeshCentral server side
it will create a self-signed certificate for the AMT https://meshcentral.myserver.com:4433

@si458 si458 added question and removed bug labels Dec 7, 2024
@k1n6b0b
Copy link
Author

k1n6b0b commented Dec 7, 2024

So the issue I'm having is none of my AMT CIRA stuff is working, maybe i'm following the wrong thread to debug

AMT: Start Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 2
AMT: pve4 Checking Intel AMT state...
AMT: pve4 Attempt Initial Contact CIRA-LMS
AMT: pve4 LMS-Connect TLS admin
AMT: pve4 Initial Contact Response 408
AMT: pve4 Attempt Initial Contact CIRA-LMS
AMT: pve4 LMS-Connect NoTLS admin
AMT: pve4 Initial Contact Response 408
AMT: pve4 Remove device node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 2 14

image image

@si458
Copy link
Collaborator

si458 commented Dec 7, 2024

you are using linux (guessing from the pve4 hostname)
intel amt and linux doesnt seem to be working correctly, the is QUITE A LOT of open issues about it
im going to try have a look next week with an v12 amt device i have and find out whats what
im the mean time, how did u setup the amt on the device to begin with?
setup a meshagent group, set amt policy to fully automatic, instal meshagent, use amtconfig in console?

#6565 #5780

@k1n6b0b
Copy link
Author

k1n6b0b commented Dec 7, 2024

Yes, Proxmox 8.3.1 / Debian 12

I tried fully automatic -- it always fell back to CCM

I also did

Rebooted host, Control-P
Changed default password 'admin' to something new

But still couldnt get out of CCM

So to get it out of CCM --

Rebooted host, Control-P
completely disabled AMT
Rebooted, Control-P
enabled AMT
Set user policy to Never
provisioned network to DHCP (it provisioned, but never got a DHCP lease)
set DNS per above

Saved/Rebooted

Note: I had set the meshcentral policy to ACM, specifying the password and telling it not to fallback to CCM by this point

Now i was able to see and edit settings with meshcmd, but I was unable to see any DHCP lease, so no IP to ping

I was able to access https://localhost:16993 and login, here I changed DHCP -> Static IP
Rebooted the host

Now I was able to access remotely to the Static IP i set https://xxx.xxx.xxx.xxx:16993 but errors on the FQDN I set in AMT config and my DNS server (which brings me to the above post, checking the cert etc)

My assumption was maybe MeshCentral was trying to access via FQDN but maybe it uses localhost, so i could have been going down an unrelated rabbithole with my initial post

Note: I do have 2 NICs in a bridge0 in proxmox -- previously i've seen issues with IPMI and VMware nic-teaming on other hardware, so noting it here
Note: this one also seemed possibly related: #6281

@si458
Copy link
Collaborator

si458 commented Dec 7, 2024

ok so

  1. Fully Automatic will try to activate ur machine into ACM mode, and if it fails it falls back to CCM, then if that fails, it just gives up - THIS IS EXPECTED!
  2. ALWAYS USE DHCP mode in the network settings, NEVER USE STATIC IP AS THIS BREAKS CIRA!!! VERY IMPORTANT!
  3. when u setup amt to begin with, i would set the dns suffix to your meshcentral dns name

@k1n6b0b
Copy link
Author

k1n6b0b commented Dec 7, 2024

    • Right, since i kept getting CCM I was trying to force ACM, makes sense
  1. Interesting.. I'm unable to get a DHCP lease -- so this is a sticking point, it only worked when set statically
  2. DNS suffix matches, as in xxx.yyy and meshcentral.xxx.yyy -- is that correct, or did you mean the dns suffix to be the FQDN of meshcentral?

@si458
Copy link
Collaborator

si458 commented Dec 7, 2024

yes unfortunately #6281 had the same issue they only have static ip ranges,
but unfortunately the Environment Detection disables CIRA working correctly if you set a STATIC IP in the AMT
this is an AMT bug and chances are will NEVER be fixed unless u bugged Intel to fix it

as for the dns suffix inside the amt network settings, it must match with whatevever your meshcentral server uses i have found
so for me i use meshcentral.mydomain.com and matches my cert value in config.json

@k1n6b0b
Copy link
Author

k1n6b0b commented Dec 7, 2024

OK I'll play around more with DHCP then. just tried it and it loses its address to 0.0.0.0

> amt
{
  core-ver: 1
  OsHostname: "pve4"
  Flags: 4
  Versions: {
    Flash: "15.0.35"
    Netstack: "15.0.35"
    AMTApps: "15.0.35"
    AMT: "15.0.35"
    Sku: "16392"
    VendorID: "8086"
    Build Number: "1951"
    Recovery Version: "15.0.35"
    Recovery Build Num: "1951"
    Legacy Mode: "False"
  }
  UUID: "2a1b708e-6c0a-dd84-7bcb-48210b37300b"
  ProvisioningMode: 1
  ProvisioningState: 2
  net0: {
    enabled: 1
    dhcpEnabled: 1
    dhcpMode: "ACTIVE"
    mac: "**48:21:0B:37:30:0B**"
    address: "0.0.0.0"
  }
}

The NUC11 has 2 NICs in a bridge

> 2: enp88s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
>     link/ether 48:21:0b:36:c2:01 brd ff:ff:ff:ff:ff:ff permaddr 48:21:0b:37:30:0b
> 3: enp89s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
>     link/ether **48:21:0b:37:30:0b** brd ff:ff:ff:ff:ff:ff permaddr 48:21:0b:36:c2:01
> 4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
>     link/ether 48:21:0b:37:30:0b brd ff:ff:ff:ff:ff:ff
> 5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
>     link/ether 48:21:0b:37:30:0b brd ff:ff:ff:ff:ff:ff
>     inet 10.0.0.24/24 scope global vmbr0
>        valid_lft forever preferred_lft forever
>     inet6 fe80::4a21:bff:fe37:300b/64 scope link 
>        valid_lft forever preferred_lft forever

@si458
Copy link
Collaborator

si458 commented Dec 7, 2024

@k1n6b0b thats correct, intel amt in dhcp mode never knows what the IP address of the device is
dont ask why, so it will always show 0.0.0.0

> amt
{
  core-ver: 1
  OsHostname: "DESKTOP-LTSAD3V"
  Flags: 4
  MeiVersion: "11.0.0.34052"
  Versions: {
    Flash: "7.1.91"
    Netstack: "7.1.91"
    AMTApps: "7.1.91"
    AMT: "7.1.91"
    Sku: "24584"
    VendorID: "8086"
    Build Number: "3272"
    Recovery Version: "7.1.91"
    Recovery Build Num: "3272"
    Legacy Mode: "False"
  }
  UUID: "4c4c4544-0031-4310-804a-c6c04f37354a"
  ProvisioningMode: 1
  ProvisioningState: 2
  net0: {
    enabled: 1
    dhcpEnabled: 1
    dhcpMode: "PASSIVE"
    mac: "18:03:73:E0:DC:04"
    address: "0.0.0.0"
  }
}

@k1n6b0b
Copy link
Author

k1n6b0b commented Dec 7, 2024

Should I see a lease in my DHCP server tho? (I do not see one)

@si458
Copy link
Collaborator

si458 commented Dec 7, 2024

Should I see a lease in my DHCP server tho? (I do not see one)

from what i recall, yes u should do when the computer is switched off,
but if u have a bond interface, the dhcp might not work because ur switch is sending the dhcp request over the bonded interfaces and when the pc is OFF, the bond isnt active

edit: so best try setting ur dhcp server to run on a single interface and see if u get a dhcp request for that interface

@k1n6b0b
Copy link
Author

k1n6b0b commented Dec 8, 2024

Interesting, at some point it pulled a lease, i see it in my DHCP server, but its inaccessible (maybe it was for a bit)

AMT: Start Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0
AMT: pve4 Checking Intel AMT state...
AMT: pve4 Attempt Initial Contact CIRA
AMT: pve4 CIRA-Connect NoTLS admin XXXXXX
AMT: pve4 Stop Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0
AMT: pve4 Remove device node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0 1
AMT: Start Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0
AMT: pve4 Checking Intel AMT state...
AMT: pve4 Attempt Initial Contact CIRA
AMT: pve4 CIRA-Connect NoTLS admin XXXXXX
AMT: pve4 Initial Contact Response 200
AMT: pve4 Intel AMT connected.
AMT: pve4 Done.

maybe its unstable... as its not accessible by ping, curl, or meshcentral tools now

AMT: pve4 Stop Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0
AMT: pve4 Remove device node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0 1
AMT: Start Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0
AMT: pve4 Checking Intel AMT state...
AMT: pve4 Attempt Initial Contact CIRA
AMT: pve4 CIRA-Connect NoTLS admin XXXXX
AMT: Start Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 2
AMT: pve4 Checking Intel AMT state...
AMT: pve4 Attempt Initial Contact CIRA-LMS
AMT: pve4 LMS-Connect TLS admin
AMT: pve4 Initial Contact Response 408
AMT: pve4 Attempt Initial Contact CIRA-LMS
AMT: pve4 LMS-Connect NoTLS admin
AMT: pve4 Initial Contact Response 408
AMT: pve4 Remove device node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 2 14
AMT: pve4 Stop Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0
AMT: pve4 Remove device node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0 1
AMT: Start Management node//IsmcdV9$fjIO4@B4IHO9jhrTb6IfxGR8TGB61kTE5mH@I@46sGmIQ58Hj6Pflox8 0
AMT: pve4 Checking Intel AMT state...
AMT: pve4 Attempt Initial Contact CIRA
AMT: pve4 CIRA-Connect NoTLS admin XXXXX

Its bouncing between showing Connectivity | Mesh Agent, Intel® AMT CIRA and nothing

@k1n6b0b
Copy link
Author

k1n6b0b commented Dec 8, 2024

rebooted the host again and its working!!

image

cannot connect to the IP leased by my DHCP server, or localhost.. i have no idea what IP its using rn but its working.

Stability def seems to be an issue, but sounds like per your note that the driver in Linux is wonky AF. I'll keep playing with it.

Let me know if i can provide anything more to help

@k1n6b0b
Copy link
Author

k1n6b0b commented Dec 8, 2024

I was also trying this, but unclear if it made any difference.

https://community.intel.com/t5/Intel-vPro-Platform/AMT-stops-responding-via-network-DHCP-IP-Address-missing/td-p/1240125/page/3

On reboot, the Current Power Mode is always auto until I change it to on

@si458
Copy link
Collaborator

si458 commented Dec 8, 2024

@k1n6b0b I'm having a similar issue at the moment with my AMT v12 machine, it seems to be flaky with its DHCP lease and gets it when it feels like it as i can't ping it and get a reply but that's for the info about power mode, I will do some investigating

@marek26340
Copy link

AMT: pve4 LMS-Connect TLS admin
AMT: pve4 Initial Contact Response 408
AMT: pve4 Attempt Initial Contact CIRA-LMS
AMT: pve4 LMS-Connect NoTLS admin

Sounds very similar to my issue. #6565
Still haven't figured this out. I think I'll just try recompiling OpenSSL so that it'll be clear of any modifications made by Canonical/Ubuntu regarding legacy TLS 1.0/1.1 support. I will report back in my own issue if I'll have any news...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants