MeshCentral Agent not signed when downloaded from Web UI

[PZan]

Describe the bug
I am attempting to sign my executables with my own code signing certificate and it seems to be working partially, but the downloads presented in the MeshCentral web ui are not actually signed, except for meshcmd.exe. I'm specifically after a signed version the meshcentral agent (background agent).

I can confirm that the files within the signedagents folder are actually signed with my certificate, but these files are not customized according to agentCustomization tag for my domain (only have one).

To Reproduce
Steps to reproduce the behavior:

First

• follow the (sub-optimal) option from Ylians video on the topic and sign your executables on the MeshCentral Server with agentsigningcert.pem on the meshcentral-data folder

Then

1. Go to 'My Account'
2. Click on the group Default
3. Click on "+ Add Agent"
4. Select OS: Windows, Installation type: Background only (or any other Windows download)
5. Download the file and check the file properties. Digital Signature tab is missing.

Expected behavior
The downloaded executable is signed with my code signing certificate

Server Software (please complete the following information):

• OS: Linux (in Azure App Services)
• Virtualization: Azure Container App
• Network: reverse proxy
• Version: 1.1.35
• Node: [e.g. 18.4.0]

Client Device (please complete the following information):

• Device: Laptop
• OS: Windows 11
• Network: WAN
• Browser: N/A

Remote Device (please complete the following information):

• Device: N/A
• OS: N/A
• Network: N/A
• Current Core Version (if known): N/A

Additional context
Add any other context about the problem here.

Your config.json file
Trying to share a dedacted copy of my config. Let me know if some essential is missing.

{
  "$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json",
  "settings": {
    "cert": "DEDACTED.URL",
    "agentPortTls": true,
    "WANonly": true,
    "port": 1234,
    "aliasPort": 443,
    "redirPort": 80,
    "redirAliasPort": 0,
    "mpsPort": 0,
    "tlsOffload": true,
    "allowLoginToken": false,
    "amtManager": false,
    "maxInvalidLogin": {
      "time": 60,
      "count": 3,
      "coolofftime": 120
    },
    "agentSignLock": true,
    "agentTimeStampServer": "http://timestamp.comodoca.com/authenticode"
  },
  "domains": {
    "": {
      "title": "DEDACTED",
      "agentKey": [
        "DEDACTED"
      ],
      "newAccounts": false,
      "certUrl": "DEDACTED",
      "userConsentFlags": {
        "desktopprompt": true,
        "terminalprompt": true,
        "fileprompt": true,
        "desktopprivacybar": true
      },
      "consentMessages": {
        "Title": "DEDACTED",
        "Desktop": "DEDACTED",
        "Terminal": "DEDACTED",
        "Files": "DEDACTED",
        "consentTimeout": 99999999,
        "autoAcceptOnTimeout": false,
        "oldStyle": true
      },
      "agentCustomization": {
        "displayName": "DEDACTED",
        "description": "DEDACTED",
        "companyName": "DEDACTED",
        "serviceName": "DEDACTED",
        "installText": "DEDACTED"
      },
      "twoFactorCookieDurationDays": 0,
      "lockAgentDownload": true,
      "mstsc": false,
      "novnc": false,
      "ssh": false,
      "passwordRequirements": {
        "loginTokens": false
      },
      "deviceMeshRouterLinks": {
        "rdp": false,
        "ssh": false,
        "scp": false,
        "extralinks": []
      },
      "allowSavingDeviceCredentials": false,
      "authStrategies": {
        "azure": {
          "callbackurl": "https://DEDACTED",
          "newAccounts": true,
          "newAccountsRights": [
            "nonewgroups",
            "notools"
          ],
          "clientid": "DEDACTED",
          "clientsecret": "DEDACTED",
          "tenantid": "DEDACTED",
          "newAccountsUserGroups": [
            "ugrp//DEDACTED"
          ]
        }
      },
      "userSessionIdleTimeout": 0,
      "userAllowedIP": [
		"DEDACTED"
      ]
    }
  }
}

[PZan]

Where on the server are the files located that are presented in the web ui? Because, as you can see I have some custom attributes to the agent in my domain and this is for sure not the same executable (nor modified version) which can be found in the signedagents folder. So in my mind this file has to be signed at the same time as the executables in the signedagents folder or otherwise the hash would be faulty (ie if it would've been a modified copy of a signed file).

[si458]

right so the idea is to do the following.

1. backup meshcentral-data !!!!
2. delete the folders signedagents and agents from inside of meshcentral-data if they are present
3. restart meshcentral and wait about 2mins before moving to next step
4. meshcentral will then generate new agents which are customised and code-signed with a self-signed certicate into signedagents
5. download the exes from signedagents to your local computer
6. code-sign them however you want (i use EZSignIt but this doesnt support hardware keys)
7. create a folder in meshcentral-data called agents
8. upload your code-signed exes into agents
9. restart meshcentral
10. redownload the meshagent from the web ui and they should be customised/code-signed how you want them

[PZan]

Awesome, thank you! Out of office today, but I will try this out first thing tomorrow morning!

[PZan]

Sorry, this didn't work. On step 4 you're mentioning that files in signedagents folder are now customised, but it doesn't appear so to me. They have the default naming convention and default text and colour scheme, which all I have made changes to in my config.json.

Another weird discovery I have made is that if I sign a (unsigned, but customised) file downloaded from the web ui, it no longer is customised after having it signed. I've tried both the authenticode-js app and Microsoft's SignTool.exe with the same outcome.

Could this issue perhaps be specific to to the Docker image?

[si458]

@PZan no i don't think it's Docker related
Also didn't realise u was using docker at all!
Have u made sure the meshcentral-data folder persists across reboots?
Can u share ur docker-compose? Or run commands?

[PZan]

Sorry, I'm not entirely sure you can actually call it Docker. It's an Azure Container App, so to my knowledge it's built on Kubernetes and so supports Docker-based containers.

Nonetheless all the necessary mappings are there and appears to be working as expected otherwise.

[PZan]

Also there's not an actual compose file, rather a set of pages where you configure the various parts of your app, if that makes sense. But basically we have configured the volumes mappings, and some Web App specific service settings to make it reverse proxy, as well as connecting to our Azure Storage Account, where the actual data reside.

But I can confirm that the data persists on reboot.

If I were to enable debugging, would the ARGS environment variable be "--debug" or simply "debug"?

[PZan]

So I've spun up a docker compose in my local dev environment (WSL2) and it appears to have the same baviour. Here I have control of the backend and have managed do some more testing.

I noticed in the root of the default docker exec working dir how there's a meshcentral folder (/opt/meshcentral/meshcentral) which is not part of any of my mounted volumes (meshcentral/data, user_files, backup, web). In this directory I found the agents subfolder, so I tried replacing MeshService64.exe inside that folder with my signed copy, and executed resetserver from the console. Unfortunately this didn't change anything. The download link still has an unsigned version of the agent.

[si458]

@PZan ive just setup a docker instance here and tried it here and its working without any problems?
downloads my signed exe without any problems
if you email myself or discord me, i can connect into ur environment and have alook whats going on for you

[si458]

so after spending 2 hours battling with the stupid azure ui and getting nowhere with trying to create a VOLUME to persist the data across reboots,
i ended up using terraform instead,
and then after 2 more hours of learning it and building all values etc
i can confirm i have no problems at all with it
it reads the config.json from the azure files as expected and created custom signedagents which i download from the azure files,
then code sign them, then upload them back to agents in the azure files again, and restart the revision,
then redownload the exe, and bom, its customised and using our code-signing cert

[PZan]

Thank your for putting the effort in. I really was not expecting you to do all this. But I must be doing something wrong then. I'm certain I followed your steps to every detail, so not sure what it could be. Unfortunately I will be out of office until Monday, but I will reach out to you on discord then.

Could you please share your compose file (if that's utilized with terraform?) with me? In my local environment (with docker-compose) I followed the steps and used the example without mongodb from the docker folder in this repo.

[si458]

i didnt use any form of docker-compose file!
i simply built a terraform file (which is below) and this build the azure resources for me rather than using the web ui
hopefully its very simple for you to follow for the web ui
then i viewed the storage share using azure storage explorer OR the web ui,
downloaded config.json, made changes, uploaded it bk,
restarted the replica, then did the steps about downloading, code signing, upload, restart again

P.S: sure reach out! my email on my github profile OR si458 on all social medias and chat platforms normally haha

# Configure the Azure provider
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "4.13.0"
    }
  }

  required_version = ">= 1.1.0"
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "uksouth"
}

resource "azurerm_log_analytics_workspace" "example" {
  name                = "acctest-01"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  sku                 = "PerGB2018"
  retention_in_days   = 30
}

resource "azurerm_container_app_environment" "example" {
  name                       = "Example-Environment"
  location                   = azurerm_resource_group.example.location
  resource_group_name        = azurerm_resource_group.example.name
  log_analytics_workspace_id = azurerm_log_analytics_workspace.example.id
}

resource "azurerm_storage_account" "example" {
  location                 = azurerm_resource_group.example.location
  resource_group_name      = azurerm_resource_group.example.name
  name                     = "examplestorageaccount123"
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_storage_share" "example" {
  name               = "example-share"
  quota              = 10
  storage_account_id = azurerm_storage_account.example.id
  enabled_protocol   = "SMB"
}

resource "azurerm_container_app_environment_storage" "example" {
  name                         = "mycontainerappstorage"
  container_app_environment_id = azurerm_container_app_environment.example.id
  account_name                 = azurerm_storage_account.example.name
  share_name                   = azurerm_storage_share.example.name
  access_key                   = azurerm_storage_account.example.primary_access_key
  access_mode                  = "ReadWrite"
}

resource "azurerm_container_app" "example" {
  name                         = "example-app"
  container_app_environment_id = azurerm_container_app_environment.example.id
  resource_group_name          = azurerm_resource_group.example.name
  revision_mode                = "Single"

  template {
    container {
      name   = "examplecontainerapp"
      image  = "ghcr.io/ylianst/meshcentral:latest"
      cpu    = 2.0
      memory = "4Gi"
      env {
        name  = "TZ"
        value = "Europe/London"
      }
      volume_mounts {
        name = "my-azure-file-volume"
        path = "/opt/meshcentral/meshcentral-data"
      }
    }
    min_replicas = 1
    max_replicas = 1
    volume {
      name         = "my-azure-file-volume"
      storage_name = azurerm_container_app_environment_storage.example.name
      storage_type = "AzureFile"
    }
  }

  ingress {
    target_port = 443
    traffic_weight {
      percentage      = 100
      latest_revision = true
    }
    external_enabled = true
  }
}