Compiled object file extension for test and release builds

[kozmotronik]

Hi!
Is it possible to specify different extensions for release and test builds?
I use the native compiler (GCC on Linux) for the test builds and it requires *.o object files for the linkage process. I use Microchip's XC8 compiler for the release builds and it requires *.p1 for the linkage.

If I define extentions below the :extension: setting in the yaml file, ceedling uses the same extensions for both test and release builds. So my question is: Could I specify different extensions so that each build use its own specification?

I've tried defining the extensions as following:

:extension:
  :test:
    :object: .o
    :executable: .out
    :assembly: .s
  :release:
    :object: .p1 # xc8 derlenmiş dosya (ara dosya) biçimi uzantısı
    :executable: .elf
    :assembly: .S

But when I release build with XC8 compiler, ceedling somehow looks for *.o files instead of *.p1 files.
Here is a sample release build output:

$ ceedling release


Release build 'hkbmv2.hex'
--------------------------
Compiling eusart2.c...
Compiling memory.c...
Compiling eusart1.c...
Compiling interrupt.c...
Compiling osilator.c...
Compiling pin.c...
Compiling devcfg.c...
Compiling hkbmv2.c...
Compiling sistimer.c...
Compiling gorev.c...
Compiling uckuyruk.c...
Compiling port.c...
Compiling araclar.c...
Compiling atio.c...
Compiling btio.c...
Compiling duduk.c...
Compiling eepromio.c...
Compiling eylem.c...
Compiling gosterge.c...
Compiling kanallar.c...
Compiling kayit.c...
Compiling kontrol.c...
Compiling tilt.c...
Compiling tus.c...
Compiling yetki.c...
Linking hkbmv2.hex...
HERE
ERROR: Shell command failed.
> Shell executed command:
'xc8-cc -mcpu=18F46K22 -Wl,-Map="build/release/hkbmv2.map" -fno-short-double -fno-short-float -O1 -fasmfile -xassembler-with-cpp -I"src" -I"src/donanim" -I"src/gorevci" -I"src/gorevci/include" -I"src/gorevci/portlar" -I"src/gorevci/portlar/mplabx" -I"src/gorevci/portlar/mplabx/xc8" -I"src/gorevci/portlar/mplabx/xc8/pic18" -I"src/uygulama" -I"src" -I"src/donanim" -I"src/gorevci/include" -I"src/gorevci/portlar/mplabx/xc8/pic18" -I"src/uygulama" -mwarn=-1 -Wa,-a -msummary=-psect,-class,+mem,-hex,-file -ginhx32 -Wl,--data-init -mno-keep-startup -mno-download -mdefault-config-bits -mstack=compiled:auto:auto:auto -o "build/release/hkbmv2.hex" build/release/out/hkbmv2.o build/release/out/devcfg.o build/release/out/eusart1.o build/release/out/eusart2.o build/release/out/interrupt.o build/release/out/memory.o build/release/out/osilator.o build/release/out/pin.o build/release/out/sistimer.o build/release/out/gorev.o build/release/out/uckuyruk.o build/release/out/port.o build/release/out/araclar.o build/release/out/atio.o build/release/out/btio.o build/release/out/duduk.o build/release/out/eepromio.o build/release/out/eylem.o build/release/out/gosterge.o build/release/out/kanallar.o build/release/out/kayit.o build/release/out/kontrol.o build/release/out/tilt.o build/release/out/tus.o build/release/out/yetki.o'
> Produced output:
::: error: (141) can't open intput file "build/release/out/hkbmv2.o": No such file or directory
(908) exit status = 1
> And exited with status: [pid 36265 exit 1].

ShellExecutionException ==> 'Microchip Xc8 Linker' (xc8-cc) exited with an error

Ceedling could not complete the build because of errors.

And here is my yaml configuration for release build:

:tools:
  :release_compiler:
    :name: "Microchip XC8 Compiler"
    :executable: xc8-cc
    # :stderr_redirect: auto
    :arguments:
      - "-mcpu=#{ENV['MCU']}"
      - -c
      - -fno-short-double
      - -fno-short-float
      - -O1
      - -fasmfile
      - -xassembler-with-cpp
      # - -D"$": COLLECTION_DEFINES_RELEASE_AND_VENDOR
      - -I"$": COLLECTION_PATHS_SOURCE_AND_INCLUDE
      - -mwarn=-1
      - -Wa,-a
      - -msummary=-psect,-class,+mem,-hex,-file
      - -ginhx32
      - -Wl,--data-init
      - -mno-keep-startup
      - -mno-download
      - -mdefault-config-bits
      - -std=c99
      - -mstack=compiled:auto:auto:auto
      - -o "${2}"
      - "${1}"

  :release_assembler:
    :name: "Microchip XC8 Compiler"
    :executable: xc8-cc
    :arguments:
      - "-mcpu=#{ENV['MCU']}"
      - -std=c99
      - -O1
      - -Wa,-a
      # - -D"$": COLLECTION_DEFINES_RELEASE_AND_VENDOR
      - -I"$": COLLECTION_PATHS_SOURCE_AND_INCLUDE
      - -o "${2}"
      - -c
      - "${1}"

  :release_linker:
    :name: "Microchip XC8 Linker"
    :executable: xc8-cc
    :arguments:
      - "-mcpu=#{ENV['MCU']}"
      - -Wl,-Map="${3}"
      - -fno-short-double
      - -fno-short-float
      - -O1
      - -fasmfile
      - -xassembler-with-cpp
      # - -D"$": COLLECTION_DEFINES_RELEASE_AND_VENDOR
      - -I"$": COLLECTION_PATHS_SOURCE_AND_INCLUDE
      - -mwarn=-1
      - -Wa,-a
      - -msummary=-psect,-class,+mem,-hex,-file
      - -ginhx32
      - -Wl,--data-init
      - -mno-keep-startup
      - -mno-download
      - -mdefault-config-bits
      - -mstack=compiled:auto:auto:auto
      - -o "${2}"
      - "{'${1}'.split(' ').map { |x| (x.include? '/out/c/') ? (Pathname(x).sub_ext('.p1')).to_s : x }.join(' ') }"
      - "${4}"

Thanks in advance, with respect.

[mvandervoord]

Hi.

At the moment, this isn't an option.

However, we are close to the next release, and this is an easy detail for us to add. I'll add it to our list.

Secondly, to work around it for now, you could create a release.yml file which has the tools and modifications for your release builds, and you could call ceedling config:release release when you want to build your releases. It's not ideal, but it'll get you running today.

Mark

[kozmotronik]

Thank you for your quick response.

I forgot to mention that I was using one of the 0.32.0 releases of ceedling version.
I will try your work around. I remember now, I have read something like this work around in one of the docs but don't remember which.

[kozmotronik]

Hi @mvandervoord!
Sorry for bothering, but I really need to complete the project unit testing as soon as possible. As a matter of fact, the unit testing is almost done if I don't count the release configuration. Because the ceedling config:release release command fails and I couldn't figure out what should I do. Here is an example output of failed execution:

$ ceedling config:release release
rake aborted!
Don't know how to build task 'config:release' (See the list of available tasks with `rake --tasks`)
(See full trace by running task with --trace)
Ceedling could not complete the build because of errors.

As per your suggestion, I've created a release.yml file. Moved all release related configuration to it and here is how it looks:
release.yml

:release_build:
 :output: hkbmv2.hex
 :use_assembly: TRUE

:environment:
  - :prjname: hkbmv2
  - :mcu: 18F46K22
  - :mplabx_root: "/opt/microchip/mplabx/v6.15"
  - :path:
    - "/opt/microchip/xc8/v2.46/bin"
    - "#{ENV['PATH']}"
    - build

:extension:
  :object: .p1
  :executable: .elf
  :assembly: .S

:tools:
  :release_compiler:
    :name: "Microchip XC8 Compiler"
    :executable: xc8-cc
    # :stderr_redirect: auto
    :arguments:
      - "-mcpu=#{ENV['MCU']}"
      - -c
      - -fno-short-double
      - -fno-short-float
      - -O1
      - -fasmfile
      - -xassembler-with-cpp
      # - -D"$": COLLECTION_DEFINES_RELEASE_AND_VENDOR
      - -I"$": COLLECTION_PATHS_SOURCE_AND_INCLUDE
      - -mwarn=-1
      - -Wa,-a
      - -msummary=-psect,-class,+mem,-hex,-file
      - -ginhx32
      - -Wl,--data-init
      - -mno-keep-startup
      - -mno-download
      - -mdefault-config-bits
      - -std=c99
      - -mstack=compiled:auto:auto:auto
      - -o "${2}"
      - "${1}"

  :release_assembler:
    :name: "Microchip XC8 Compiler"
    :executable: xc8-cc
    :arguments:
      - "-mcpu=#{ENV['MCU']}"
      - -std=c99
      - -O1
      - -Wa,-a
      # - -D"$": COLLECTION_DEFINES_RELEASE_AND_VENDOR
      - -I"$": COLLECTION_PATHS_SOURCE_AND_INCLUDE
      - -o "${2}"
      - -c
      - "${1}"

  :release_linker:
    :name: "Microchip XC8 Linker"
    :executable: xc8-cc
    :arguments:
      - "-mcpu=#{ENV['MCU']}"
      - -Wl,-Map="${3}"
      - -fno-short-double
      - -fno-short-float
      - -O1
      - -fasmfile
      - -xassembler-with-cpp
      # - -D"$": COLLECTION_DEFINES_RELEASE_AND_VENDOR
      - -I"$": COLLECTION_PATHS_SOURCE_AND_INCLUDE
      - -mwarn=-1
      - -Wa,-a
      - -msummary=-psect,-class,+mem,-hex,-file
      - -ginhx32
      - -Wl,--data-init
      - -mno-keep-startup
      - -mno-download
      - -mdefault-config-bits
      - -mstack=compiled:auto:auto:auto
      - -o "${2}"
      - "{'${1}'.split(' ').map { |x| (x.include? '/out/c/') ? (Pathname(x).sub_ext('.p1')).to_s : x }.join(' ') }"
      - "${4}"

Now, the project.yml file is free of any release related configuration, hence its default tool chain is GCC (native).
Could you please point out what I am doing wrong? Thank you very much!

[JuPrgn]

Hi,

I am using XC16 with two project files : project.yml and project_XC16.yml and I run my tests with ceedling project:project_XC16 test:all. I am not really using the release feature, but maybe my configuration files can be helpful for you ?

I have always been using project, but config should be similar. Perhaps project is an alias, but I cannot find it in the documentation anymore. Anyway this is working using project and I get errors with config but I am on Ceedling 0.31.0.

This is the content of my project.yml :

---
# Default configuration, defines the common parameters 
# shared with XC16 and GCC configurations
#
:project:
  :use_exceptions: FALSE
  :use_test_preprocessor: TRUE
  :use_deep_dependencies: TRUE
  :use_auxiliary_dependencies: TRUE
  :build_root: build_test
  :test_file_prefix: test_
  :which_ceedling: vendor/ceedling
  :default_tasks:
    - test:all

:extension:
  :executable: .hex

:paths:
  :test:
    - +:test/**
    - -:test/support
  :source:
    - src/**
  :support:
    - test/support  
  :include:
    - test/support 
    - mcc_generated_files
    - src/bsp
    - plugin

:defines:
  :commmon: &common_defines
    - UNITY_INT_WIDTH=16
    - CMOCK_MEM_INDEX_TYPE=uint16_t
    # - CMOCK_MEM_PTR_AS_INT=uint16_t # segmentation fault with GCC when using ExpectAndReturn if specified
    - CMOCK_MEM_ALIGN=1
    - CMOCK_MEM_SIZE=4096
    - __dsPIC33EP256MC506__
    - CO_FCY=64000
    - CO_VERSION_MAJOR=4
    - CO_USE_GLOBALS
    - CO_CONFIG_SRDO=0
    - CO_CONFIG_TRACE=0
  :test:
    - *common_defines
    - TEST
    # - __eds__=
  :test_preprocess:
    - *common_defines
    - TEST

:cmock:
  :mock_prefix: mock_
  :when_no_prototypes: :warn
  :enforce_strict_ordering: TRUE
  :plugins:
    - :ignore
    - :callback
  :treat_as:
    uint8:    HEX8
    uint16:   HEX16
    uint32:   UINT32
    uint64:   UINT64
    int8:     INT8
    bool:     UINT8

# Add -gcov to the plugins list to make sure of the gcov plugin
# You will need to have gcov and gcovr both installed to make it work.
# For more information on these options, see docs in plugins/gcov
:gcov:
  :html_report: TRUE
  :html_report_type: detailed
  :html_medium_threshold: 75
  :html_high_threshold: 90
  :xml_report: FALSE
  :reports:
    - SonarQube

# LIBRARIES
# These libraries are automatically injected into the build process. Those specified as
# common will be used in all types of builds. Otherwise, libraries can be injected in just
# tests or releases. These options are MERGED with the options in supplemental yaml files.
:libraries:
  :placement: :end
  :flag: "${1}"  # or "-L ${1}" for example
  :common: &common_libraries []
  :test:
    - *common_libraries
  :release:
    - *common_libraries

:plugins:
  :load_paths:
    - vendor/ceedling/plugins
  :enabled:
    - stdout_pretty_tests_report
    - module_generator
    - raw_output_report
    
:module_generator:
  # use template files because direct boilerplates are not interpreted as UTF-8
  :boilerplate_files:
    :src: 'template\src_boilerplate.txt'
    :inc: 'template\inc_boilerplate.txt'
    :tst: 'template\tst_boilerplate.txt'
...

And project_XC16.yml :

---
# Specific configuration to run on Windows using XC16
# these options are added to default project.yml
#
# Required : Install XC16 and configure XC16PATH
#
:environment:
  - :mcpu: 33EP256MC506
  - :XC16PATH: "C:/Program Files/Microchip/xc16/v1.70"
  - :XC16GCC: "#{ENV['XC16PATH']}/bin/xc16-gcc.exe"
  - :XC16P1: "#{ENV['XC16PATH']}/support/dsPIC33E/h"
  - :XC16P2: "#{ENV['XC16PATH']}/support/generic/h"
  - :XC16P3: "#{ENV['XC16PATH']}/include/lega-c"
    
:paths:
  :include:
    - "#{ENV['XC16P1']}"
    - "#{ENV['XC16P2']}"
    - "#{ENV['XC16P3']}"
    
:tools:
  :test_dependencies_generator:
    :executable: "#{ENV['XC16GCC']}"
    :name: 'XC 16 Test Dependencies Generator'
    :arguments:
      - -E
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
      - -D$: COLLECTION_DEFINES_TEST_AND_VENDOR
      - -D$: DEFINES_TEST_PREPROCESS
      - -DGNU_COMPILER
      - -MT "${3}"
      - -MM
      - -MD
      - -MG
      - -MF"${2}"
      - -c "${1}"
  :test_includes_preprocessor:
    :executable: "#{ENV['XC16GCC']}"
    :name: 'XC 16 Test Include Preprocessor'
    :arguments:
      - -mcpu=#{ENV['MCPU']}
      - -E
      - -MM
      - -MG
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
      - -D$: COLLECTION_DEFINES_TEST_AND_VENDOR
      - -D$: DEFINES_TEST_PREPROCESS
      - -DGNU_COMPILER
      - "${1}"
  :test_file_preprocessor:
    :executable: "#{ENV['XC16GCC']}"
    :name: 'XC 16 Test File Preprocessor'
    :arguments:
      - -mcpu=#{ENV['MCPU']}
      - -E
      - -DGNU_COMPILER
      - "${1}"
      - -o "${2}"
      - -D$: COLLECTION_DEFINES_TEST_AND_VENDOR
      - -D$: DEFINES_TEST_PREPROCESS
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
  :test_file_preprocessor_directives:
    :executable: "#{ENV['XC16GCC']}"
    :name: 'XC 16 Test File Preprocessor Directives'
    :arguments:
      - -mcpu=#{ENV['MCPU']}
      - -E
      - -DGNU_COMPILER
      - "${1}"
      - -o "${2}"
      - -D$: COLLECTION_DEFINES_TEST_AND_VENDOR
      - -D$: DEFINES_TEST_PREPROCESS
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
      - -fdirectives-only
  :test_compiler:
    :executable: "#{ENV['XC16GCC']}"
    :arguments:
      - -mcpu=#{ENV['MCPU']}
      - -x c
      - -omf=elf
      - -msmart-io=1
      - -mno-eds-warn  
      - -msfr-warn=off
      - -legacy-libc
      - -c
      - "${1}"
      - -o "${2}"
      - -D$: COLLECTION_DEFINES_TEST_AND_VENDOR
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -Wall
      - -mlarge-code
      - -mlarge-data
      - -mlarge-scalar
      - -mconst-in-data
      - -std=gnu99
  :test_linker:
    :executable: "#{ENV['XC16GCC']}"
    :arguments:
      - -mcpu=#{ENV['MCPU']}
      - -omf=elf 
      - -DXPRJ_default=default  
      - -legacy-libc 
      - ${1}
      - -o "./build_test/release/TestBuild.out"
      - -Wl,--allow-multiple-definition,-Tp#{ENV['MCPU']}.gld,--heap=0,--stack=16,--local-stack,--check-sections,--data-init,--pack-data,--handles,--isr,--no-gc-sections,--fill-upper=0,--stackguard=16,--no-force-link,--smart-io,-Map=./build_test/release/TestOutput.map,--report-mem,--memorysummary,./build_test/release/memoryfile.xml
  :test_fixture:
    :executable: ruby
    :name: "Microchip simulator test fixture"
    :stderr_redirect: :win #inform Ceedling what model of $stderr capture to use
    :arguments:
      - test/simulation/sim_test_fixture.rb
  :release_compiler:
    :executable: "#{ENV['XC16GCC']}"
    :arguments:
      - -mcpu=#{ENV['MCPU']}
      - -x c
      - -omf=hex
      - -msmart-io=1
      - -mno-eds-warn  
      - -msfr-warn=off
      - -legacy-libc
      - -c
      - "${1}"
      - -o "${2}"
      - -I"$": COLLECTION_PATHS_SOURCE_INCLUDE_VENDOR
      - -I"$": COLLECTION_PATHS_RELEASE_TOOLCHAIN_INCLUDE
      - -D$: COLLECTION_DEFINES_RELEASE_AND_VENDOR
      - -Wall
      - -mlarge-code
      - -mlarge-data
      - -mlarge-scalar
      - -mconst-in-data
  :release_linker:
    :executable: "#{ENV['XC16GCC']}"
    :arguments:
      - -mcpu=#{ENV['MCPU']}
      - -legacy-libc
      - ${1}
      - -o "${2}"
      - -Wl,-Tp#{ENV['MCPU']}.gld,--heap=0,--stack=16,--local-stack,--check-sections,--data-init,--pack-data,--handles,--isr,--no-gc-sections,--fill-upper=0,--stackguard=16,--no-force-link,--smart-io,-Map=./build_test/release/ReleaseOutput.map,--report-mem,--memorysummary,./build_test/release/memoryfile.xml
...

[kozmotronik]

Hi @JuPrgn,
Thank you for your response. That was it, it worked!
By modifying the suggestion of @mvandervoord which was ceedling config:release release to ceedling project:release release as per your tip, release compilation was successful.

Thank you very much for the tip!

[JuPrgn]

You are welcome ! 😉
I don't know why project is no more documented maybe @mvandervoord can give use some information on this and how to use config instead ? I have an open PR to add multi project feature to VSCode Ceedling plugin and have to figure it out to work on this. numaru/vscode-ceedling-test-adapter#128