Segmentation Fault when multi-part upload through proxy

[hjia97]

Describe the bug

Hello! I want to ask and report an issue I encountered during multi-thread upload of Boto3.

I can reproduce the crash consistently on my Mac environment, and also an Alpine 3.20 Docker container.

I have an individual Python script that tries to multi-part upload a large file to S3 THROUGH a proxy. The thread count is set to be 40, or any large number.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

My python program would crash instantly.

zsh: segmentation fault  python3 upload.py

Current Behavior

I used lldb to track down to the exact broken point. It looks like OPENSSL library that's used by Boto3 is freeing a memory location that has already be freed.

Reproduction Steps

The code I used:

boto_session = boto3.Session(aws_session_token=aws_session_token, aws_access_key_id=aws_access_key_id,
                             aws_secret_access_key=aws_secret_access_key)

transfer_config = {
    'multipart_threshold': THRESHOLD,
    'multipart_chunksize': CHUNKSIZE,
    'max_concurrency': 40,
    'use_threads': True}

tc = TransferConfig(max_bandwidth=MAX_BANDWITDTH, **transfer_config)

client_config = Config(endpoint_discovery_enabled=True,
                       proxies={'https': 'https://' + proxy},
                       proxies_config={
                           'proxy_client_cert': 'cert.pem',
                           'proxy_use_forwarding_for_https': True,
                           'proxy_ca_bundle': 'bundle.crt'
                       },
                       client_cert='cert.pem')

client = boto_session.client('s3', use_ssl=True, verify='ion_dev_certificate.pem',
                             config=client_config,
                             region_name=region_name)

client.upload_file(file_path, bucket, dest_path, Config=tc)

Possible Solution

No response

Additional Information/Context

The crash could be recreated consistently at my Mac, and a clean Alpine 3.20 Docker container.

SDK version used

[email protected],

Environment details (OS name and version, etc.)

Alpine3.20, Mac