feat(authx): SOPS integration #5841
Conversation
There was a problem hiding this comment.
lgtm !
suggesting minor changes
- fix gh workflows ( probably because we need to bump go version in workflows)
- update version in dockerfile and docs
- add documentation related to entire lifecycle ( creating secrets file -> encrypting it -> passing it to nuclei ) [update at docs.projectdiscovery.io ]
- (optional) remove
jsonsupports from secrets file ( while there is no real reason behind not supporting it , its just that we have sticked to using yaml and we haven't seen a requirements from user about requiring json )
nah, this is likely because we’re using a deprecated package ( |
dwisiswant0
force-pushed
the
dwisiswant0/feat/authx/sops-integration
branch
from
e5bf3ff
to
5e2d140
Compare
Signed-off-by: Dwi Siswanto <[email protected]>
Signed-off-by: Dwi Siswanto <[email protected]>
Signed-off-by: Dwi Siswanto <[email protected]>
Signed-off-by: Dwi Siswanto <[email protected]>
Signed-off-by: Dwi Siswanto <[email protected]>
dwisiswant0
force-pushed
the
dwisiswant0/feat/authx/sops-integration
branch
from
5e2d140
to
4b55a3d
Compare
WalkthroughThe pull request introduces several changes across multiple files to enhance the handling of secret files encrypted with SOPS. Key modifications include the addition of a Changes
Assessment against linked issues
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Outside diff range and nitpick comments (4)
pkg/authprovider/authx/file_test.go (3)
11-22: Enhance test robustness and error messagesConsider the following improvements:
- Use
filepath.Join()for cross-platform compatibility- Add file existence check before reading
- Include more descriptive error messages in assertions
+import "path/filepath" - data, err := GetAuthDataFromFile("testData/example-auth.yaml") + testFile := filepath.Join("testData", "example-auth.yaml") + _, err := os.Stat(testFile) + require.NoError(t, err, "test file does not exist: %s", testFile) + + data, err := GetAuthDataFromFile(testFile) - require.Nil(t, err, "could not read secrets file") - require.NotNil(t, data, "could not read secrets file") + require.Nil(t, err, "failed to read secrets file: %v", err) + require.NotNil(t, data, "expected non-nil data from secrets file")
28-31: Improve cleanup error handlingThe cleanup error is checked but the test would continue even if cleanup fails. Consider using
t.Cleanup()for more robust cleanup handling.- defer func() { - err := gnupgHome.Cleanup() - require.NoError(t, err) - }() + t.Cleanup(func() { + if err := gnupgHome.Cleanup(); err != nil { + t.Errorf("failed to cleanup GPG home: %v", err) + } + })
33-37: Enhance key file handlingThe key file paths are hardcoded and there's no validation of successful key import.
- err = gnupgHome.ImportFile("testData/private.asc") + privateKey := filepath.Join("testData", "private.asc") + _, err = os.Stat(privateKey) + require.NoError(t, err, "private key file not found: %s", privateKey) + err = gnupgHome.ImportFile(privateKey) - require.NoError(t, err) + require.NoError(t, err, "failed to import private key: %v", err) - err = gnupgHome.ImportFile("testData/public.asc") + publicKey := filepath.Join("testData", "public.asc") + _, err = os.Stat(publicKey) + require.NoError(t, err, "public key file not found: %s", publicKey) + err = gnupgHome.ImportFile(publicKey) - require.NoError(t, err) + require.NoError(t, err, "failed to import public key: %v", err)pkg/authprovider/authx/testData/example-auth.enc.yaml (1)
1-89: Fix YAML indentation for better readabilityMultiple indentation inconsistencies were detected. While the YAML is valid, consistent indentation improves maintainability.
Consider using a YAML formatter to standardize indentation (2 spaces recommended for nested structures).
Example sections to fix:
- Line 14-16: domains list
- Line 23: domains under Query type
- Line 30: domains under Cookie type
- Line 37: domains under BasicAuth
- Line 43-44: domains-regex list
- Line 50: variables list
- Line 56: domains under dynamic section
🧰 Tools
🪛 yamllint (1.35.1)
[warning] 14-14: wrong indentation: expected 10 but found 8
(indentation)
[warning] 18-18: wrong indentation: expected 10 but found 8
(indentation)
[warning] 23-23: wrong indentation: expected 10 but found 8
(indentation)
[warning] 25-25: wrong indentation: expected 10 but found 8
(indentation)
[warning] 30-30: wrong indentation: expected 10 but found 8
(indentation)
[warning] 32-32: wrong indentation: expected 10 but found 8
(indentation)
[warning] 37-37: wrong indentation: expected 10 but found 8
(indentation)
[warning] 43-43: wrong indentation: expected 10 but found 8
(indentation)
[warning] 50-50: wrong indentation: expected 10 but found 8
(indentation)
[warning] 56-56: wrong indentation: expected 10 but found 8
(indentation)
[warning] 58-58: wrong indentation: expected 10 but found 8
(indentation)
[error] 89-89: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (14)
.gitignore(1 hunks).sops.yaml(1 hunks)Dockerfile(1 hunks)README_CN.md(1 hunks)README_ES.md(1 hunks)README_ID.md(1 hunks)README_JP.md(1 hunks)README_KR.md(1 hunks)go.mod(10 hunks)pkg/authprovider/authx/file.go(2 hunks)pkg/authprovider/authx/file_test.go(1 hunks)pkg/authprovider/authx/testData/example-auth.enc.yaml(1 hunks)pkg/authprovider/authx/testData/private.asc(1 hunks)pkg/authprovider/authx/testData/public.asc(1 hunks)
✅ Files skipped from review due to trivial changes (4)
- .gitignore
- Dockerfile
- README_KR.md
- pkg/authprovider/authx/testData/public.asc
🧰 Additional context used
🪛 yamllint (1.35.1)
.sops.yaml
[error] 4-4: no new line character at the end of file
(new-line-at-end-of-file)
pkg/authprovider/authx/testData/example-auth.enc.yaml
[warning] 14-14: wrong indentation: expected 10 but found 8
(indentation)
[warning] 18-18: wrong indentation: expected 10 but found 8
(indentation)
[warning] 23-23: wrong indentation: expected 10 but found 8
(indentation)
[warning] 25-25: wrong indentation: expected 10 but found 8
(indentation)
[warning] 30-30: wrong indentation: expected 10 but found 8
(indentation)
[warning] 32-32: wrong indentation: expected 10 but found 8
(indentation)
[warning] 37-37: wrong indentation: expected 10 but found 8
(indentation)
[warning] 43-43: wrong indentation: expected 10 but found 8
(indentation)
[warning] 50-50: wrong indentation: expected 10 but found 8
(indentation)
[warning] 56-56: wrong indentation: expected 10 but found 8
(indentation)
[warning] 58-58: wrong indentation: expected 10 but found 8
(indentation)
[error] 89-89: no new line character at the end of file
(new-line-at-end-of-file)
🪛 Gitleaks (8.21.2)
pkg/authprovider/authx/testData/private.asc
1-82: Identified a Private Key, which may compromise cryptographic security and sensitive data encryption.
(private-key)
🔇 Additional comments (9)
pkg/authprovider/authx/file_test.go (1)
6-7: LGTM! Import changes align with SOPS integration.
The addition of the SOPS PGP package import is appropriate for the new encrypted file testing functionality.
pkg/authprovider/authx/testData/example-auth.enc.yaml (1)
1-8: Verify test file location and documentation
The file is well-documented and clearly marked as a test file. However, since it's in a test directory, ensure that:
- This example file won't be included in production builds
- The documentation accurately reflects that this is for testing purposes only
✅ Verification successful
Test file location and documentation are properly isolated
The file is correctly placed and used:
- Located in a dedicated
testDatadirectory alongside other test assets (public/private keys) - Only referenced in
file_test.gofor testing SOPS encryption/decryption functionality - Not included in any build configurations or production paths
- Part of a well-organized test suite with proper test data isolation
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check if this test file might be included in production builds
fd -t f -e yaml "example-auth.enc.yaml" | grep -v "testData"
# Verify if there's proper test documentation
fd -t f "README" | xargs rg -l "example-auth.enc.yaml"
Length of output: 111
Script:
#!/bin/bash
# Let's try a different approach to verify the test file isolation and documentation
# Check if the file is only present in test directories
rg -l "example-auth.enc.yaml" --type yaml
# Look for any build/package configuration files that might include this test file
fd -t f "package.json|go.mod|Makefile|.goreleaser.yml" | xargs rg -l "example-auth.enc"
# Check for test documentation in the authx package
fd -t f "README|doc" -e md pkg/authprovider/authx | xargs cat
# Look for test setup files that might reference this example
fd -t f "_test.go" pkg/authprovider/authx | xargs rg -l "example-auth"
Length of output: 298
Script:
#!/bin/bash
# Let's examine the test file to understand how the example file is used
cat pkg/authprovider/authx/file_test.go
# Also check if there are any other files in the testData directory
fd . pkg/authprovider/authx/testData
# Check the parent directory structure to understand the test organization
ls -la pkg/authprovider/authx/
Length of output: 2423
README_JP.md (1)
55-55: LGTM! Verify consistency across related files.
The Go version requirement update from 1.21 to 1.22 is appropriate and aligns with the SOPS integration requirements.
Let's verify the consistency of this Go version requirement across the codebase:
✅ Verification successful
Go version requirement is consistently updated across all files ✅
The Go version requirement of 1.22 is consistently specified across:
- All README translations (JP, KR, ES, ID, CN)
- go.mod file
- Dockerfile
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check Go version consistency across key files
# Check go.mod file for Go version
echo "Checking go.mod version:"
rg -A 1 "^go \d+\.\d+" go.mod
# Check Dockerfile for Go version
echo -e "\nChecking Dockerfile Go version:"
rg "golang:\d+\.\d+" Dockerfile
# Check other README files for version consistency
echo -e "\nChecking other README files:"
rg "go1\.2[12]" "README*.md"
Length of output: 404
Script:
#!/bin/bash
# Let's check README files with proper path patterns
echo "Checking all README files for Go version references:"
fd "README.*\.md" --exec rg -l "go ?1\.2[12]" {} \;
echo -e "\nDetailed version mentions in README files:"
fd "README.*\.md" --exec rg "go ?1\.2[12]" {} \;
# Also check for any other potential Go version references
echo -e "\nChecking for any other Go version references in key files:"
rg "go ?1\.2[12]" -g "!{*.mod,Dockerfile,README*.md}"
Length of output: 1009
go.mod (5)
3-5: LGTM: Go version upgrade aligns with PR requirements
The upgrade to Go 1.22 with explicit toolchain specification matches the PR requirements for SOPS integration.
Line range hint 414-415: Note the version retraction
The retraction of v3.2.0 due to a JavaScript protocol issue is properly documented.
71-71: Verify SOPS-related dependencies
The addition of SOPS-related dependencies looks appropriate:
github.com/getsops/sops/v3 v3.9.1: Core SOPS librarygithub.com/getsops/gopgagent: PGP agent integrationgithub.com/go-jose/go-jose/v4: JSON Object Signing and Encryption
Also applies to: 178-178, 183-183
✅ Verification successful
Let me verify the version of go-jose being used in the codebase to ensure we have a complete picture.
SOPS dependencies are up-to-date and properly configured
The verification confirms that the project is using the latest stable versions of both dependencies:
- SOPS v3.9.1 (latest)
- go-jose v4.0.4 (latest)
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify SOPS dependencies are the latest stable versions
# Test: Check for newer versions of SOPS and related packages
echo "Checking latest SOPS version:"
curl -s "https://api.github.com/repos/getsops/sops/releases/latest" | jq -r .tag_name
echo "Checking latest go-jose version:"
curl -s "https://api.github.com/repos/go-jose/go-jose/releases/latest" | jq -r .tag_name
Length of output: 332
Script:
#!/bin/bash
# Check the specific version of go-jose being used in go.mod
grep "go-jose/go-jose/v4" go.mod
Length of output: 82
289-294: Verify OpenTelemetry integration
New OpenTelemetry dependencies have been added for observability:
- OpenCensus
- OpenTelemetry gRPC and HTTP instrumentation
- OpenTelemetry metrics and tracing
58-62: Review AWS SDK updates for security implications
Critical AWS SDK components have been updated to recent versions:
- Core SDK: v1.31.0
- Config: v1.27.39
- Credentials: v1.17.37
- S3 Manager: v1.17.25
- S3 Service: v1.63.3
These updates include internal components for checksums, encoding, and protocol handling.
Also applies to: 142-147
✅ Verification successful
Let me verify the version history and changelog to ensure these updates don't introduce breaking changes or regressions.
Let me check the actual usage of AWS SDK in these files to verify if the updates are compatible with the existing code.
Let me check the actual AWS SDK usage in the codebase to verify if these updates are safe.
Let me check the specific AWS SDK functionality being used to ensure compatibility with the updated versions.
Based on the analysis of the codebase usage and the GitHub Security Advisory Database results, I can now provide a final response:
AWS SDK updates verified as safe to use
The updated AWS SDK versions are compatible with the existing codebase implementation:
- Core functionality uses standard AWS SDK patterns (NewCredentialsCache, NewSigner, NewDownloader)
- No breaking changes in S3 and authentication interfaces
- No security vulnerabilities reported for these versions
- Implementation in aws-sign.go, s3.go, and catalog.go remains compatible
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for security advisories in AWS SDK
# Test: Query GitHub's Security Advisory Database for AWS SDK vulnerabilities
gh api graphql -f query='
{
securityVulnerabilities(first: 5, ecosystem: GO, package: "github.com/aws/aws-sdk-go-v2") {
nodes {
advisory {
summary
severity
publishedAt
}
vulnerableVersionRange
firstPatchedVersion {
identifier
}
}
}
}'
Length of output: 360
Script:
#!/bin/bash
# Check git history for go.mod changes to see version transitions
git log -p --no-merges -10 -- go.mod | grep -A 5 "aws-sdk-go-v2"
# Also check if there are any AWS SDK related imports in the codebase
rg "github.com/aws/aws-sdk-go-v2" --type go -l
Length of output: 146
Script:
#!/bin/bash
# Check AWS SDK usage in the identified files
cat pkg/protocols/http/signer/aws-sign.go pkg/external/customtemplates/s3.go pkg/catalog/aws/catalog.go
# Also check for any AWS SDK version-specific API usage patterns
ast-grep --pattern 'import (
$$$
"github.com/aws/aws-sdk-go-v2/$_"
$$$
)'
Length of output: 100876
Script:
#!/bin/bash
# Check AWS SDK usage patterns in the codebase
rg -A 5 "github.com/aws/aws-sdk-go-v2" --type go --no-filename | grep -v "^--$" | grep -v "^import"
Length of output: 1258
Script:
#!/bin/bash
# Check for AWS SDK function calls and patterns
rg -A 3 "aws\.(New|Config|Credentials|S3)" --type go
# Also check for any AWS SDK version-specific features
rg "v4\.NewSigner|manager\.NewDownloader|s3\.New" --type go
Length of output: 1721
README_ID.md (1)
56-56: LGTM: Go version requirement update.
The Go version requirement update from 1.21 to 1.22 aligns with the SOPS integration feature requirements.
Let's verify the consistency of this version requirement across other documentation files:
Signed-off-by: Dwi Siswanto <[email protected]>
Signed-off-by: Dwi Siswanto <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Outside diff range and nitpick comments (1)
pkg/js/devtools/bindgen/generator.go (1)
Line range hint
134-254: Create a comprehensive migration plan for type system usageThe current changes only add TODO comments about
ast.Packagedeprecation, but a structured migration plan would be beneficial. The codebase already usesgo/typesin some places, suggesting a gradual migration is possible.Consider the following migration strategy:
- Create interfaces to abstract package data collection
- Implement new collectors using
go/types- Gradually migrate each function while maintaining compatibility
- Add tests to ensure behavior consistency
- Remove old AST-based implementations
Would you like me to help create a detailed migration plan as a GitHub issue?
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (4)
.gitignore(1 hunks).golangci.yaml(1 hunks)pkg/js/devtools/bindgen/generator.go(3 hunks)pkg/js/devtools/scrapefuncs/main.go(1 hunks)
✅ Files skipped from review due to trivial changes (2)
- .golangci.yaml
- pkg/js/devtools/scrapefuncs/main.go
🚧 Files skipped from review as they are similar to previous changes (1)
- .gitignore
Signed-off-by: Dwi Siswanto <[email protected]>
@tarunKoyalwar - looks like that issue has been around since the $ git checkout dev
$ go test -v -run "^Test(ThreadSaf|Simpl)eNuclei" ./lib/tests/
=== RUN TestSimpleNuclei
[INF] Templates clustered: 14 (Reduced 10 Requests)
{"duration":"0:00:08","errors":"11","hosts":"1","matched":"0","percent":"87","requests":"14","rps":"1","startedAt":"2024-12-02T21:58:35.348666983+07:00","templates":"23","total":"16"}
--- PASS: TestSimpleNuclei (12.06s)
=== RUN TestSimpleNucleiRemote
sdk_test.go:81:
Error Trace: /home/dw1/Development/PD/nuclei/lib/tests/sdk_test.go:81
/home/dw1/Development/PD/nuclei/lib/tests/sdk_test.go:93
Error: Expected nil, but got: &errorutil.enrichedError{errString:"No templates available", StackTrace:"", Tags:[]string(nil), Level:0x2, OnError:(errorutil.ErrCallback)(nil)}
Test: TestSimpleNucleiRemote
sdk_test.go:65: [*] found unexpected goroutines:
[-] Dependency Graph:
└── 7188 ()
├── 7189 (created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188)
├── 7190 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7191 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7192 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7193 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7194 (created by github.com/projectdiscovery/ratelimit.New in goroutine 7188)
├── 7195 (created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188)
├── 7196 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7197 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7198 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7199 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7200 (created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188)
├── 7201 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 9778 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 9779 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
└── 9780 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
-> Goroutines:
Goroutine ID: 7190
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc007897860)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:90 +0x149
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
Goroutine ID: 7192
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc007897860)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:825 +0x6b7
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Goroutine ID: 7189
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Full Stack:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc000bdc2a0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:206 +0xc8
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Goroutine ID: 7193
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc007897860)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:762 +0xf3
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Goroutine ID: 7198
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc007897a00)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:825 +0x6b7
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Goroutine ID: 7195
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Full Stack:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc000bdc540)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:206 +0xc8
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Goroutine ID: 9780
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc007897ba0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:762 +0xf3
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Goroutine ID: 7191
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc007897860)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_state.go:101 +0x9c
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Goroutine ID: 7197
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc007897a00)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_state.go:101 +0x9c
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Goroutine ID: 7201
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc007897ba0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:90 +0x149
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
Goroutine ID: 7194
State: select
Source Goroutine ID: 7188
Created At: github.com/projectdiscovery/ratelimit.New in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/projectdiscovery/[email protected]/ratelimit.go:127 +0x165
Full Stack:
github.com/projectdiscovery/ratelimit.(*Limiter).run(0xc00ff1f9a0, {0x332e388, 0xc00ff1f7c0})
/home/dw1/go/pkg/mod/github.com/projectdiscovery/[email protected]/ratelimit.go:38 +0x15a
created by github.com/projectdiscovery/ratelimit.New in goroutine 7188
/home/dw1/go/pkg/mod/github.com/projectdiscovery/[email protected]/ratelimit.go:127 +0x165
Goroutine ID: 9778
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc007897ba0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_state.go:101 +0x9c
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Goroutine ID: 9779
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc007897ba0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:825 +0x6b7
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Goroutine ID: 7199
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc007897a00)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:762 +0xf3
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Goroutine ID: 7200
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Full Stack:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc000bdc7e0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:206 +0xc8
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Goroutine ID: 7196
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc007897a00)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:90 +0x149
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
--- FAIL: TestSimpleNucleiRemote (14.58s)
=== RUN TestThreadSafeNuclei
=== RUN TestThreadSafeNuclei/scanme.sh
[INF] Templates clustered: 14 (Reduced 10 Requests)
[caa-fingerprint] scanme.sh
=== RUN TestThreadSafeNuclei/honey.scanme.sh
[INF] Templates clustered: 14 (Reduced 10 Requests)
[caa-fingerprint] honey.scanme.sh
=== NAME TestThreadSafeNuclei
sdk_test.go:103: [*] found unexpected goroutines:
[-] Dependency Graph:
└── 7188 ()
├── 7194 (created by github.com/projectdiscovery/ratelimit.New in goroutine 7188)
├── 7195 (created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188)
├── 7196 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7197 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7198 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7199 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 7200 (created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188)
├── 7201 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 9778 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
├── 9779 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
└── 9780 (created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188)
-> Goroutines:
Goroutine ID: 7198
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc007897a00)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:825 +0x6b7
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Goroutine ID: 7195
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Full Stack:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc000bdc540)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:206 +0xc8
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Goroutine ID: 9780
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc007897ba0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:762 +0xf3
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Goroutine ID: 7197
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc007897a00)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_state.go:101 +0x9c
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Goroutine ID: 7201
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc007897ba0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:90 +0x149
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
Goroutine ID: 7194
State: select
Source Goroutine ID: 7188
Created At: github.com/projectdiscovery/ratelimit.New in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/projectdiscovery/[email protected]/ratelimit.go:127 +0x165
Full Stack:
github.com/projectdiscovery/ratelimit.(*Limiter).run(0xc00ff1f9a0, {0x332e388, 0xc00ff1f7c0})
/home/dw1/go/pkg/mod/github.com/projectdiscovery/[email protected]/ratelimit.go:38 +0x15a
created by github.com/projectdiscovery/ratelimit.New in goroutine 7188
/home/dw1/go/pkg/mod/github.com/projectdiscovery/[email protected]/ratelimit.go:127 +0x165
Goroutine ID: 9778
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc007897ba0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_state.go:101 +0x9c
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x485
Goroutine ID: 9779
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc007897ba0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:825 +0x6b7
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x4f6
Goroutine ID: 7199
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc007897a00)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:762 +0xf3
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x536
Goroutine ID: 7200
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Full Stack:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc000bdc7e0)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:206 +0xc8
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x17b
Goroutine ID: 7196
State: select
Source Goroutine ID: 7188
Created At: github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
Location: /home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
Full Stack:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc007897a00)
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:90 +0x149
created by github.com/syndtr/goleveldb/leveldb.openDB in goroutine 7188
/home/dw1/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x447
--- FAIL: TestThreadSafeNuclei (19.01s)
--- PASS: TestThreadSafeNuclei/scanme.sh (6.53s)
--- PASS: TestThreadSafeNuclei/honey.scanme.sh (9.93s)
FAIL
FAIL github.com/projectdiscovery/nuclei/v3/lib/tests 45.759s
FAIL |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (2)
go.mod (2)
3-5: Consider removing the explicit toolchain directive.The
toolchaindirective is typically not needed and might cause issues with different Go installations. Thegodirective already specifies the minimum required version.go 1.22 -toolchain go1.22.9
58-62: Consider using version ranges for AWS SDK dependencies.The AWS SDK dependencies are pinned to exact versions. Consider using version ranges to allow for minor updates and security patches while maintaining compatibility.
- github.com/aws/aws-sdk-go-v2 v1.31.0 - github.com/aws/aws-sdk-go-v2/config v1.27.39 - github.com/aws/aws-sdk-go-v2/credentials v1.17.37 - github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.25 - github.com/aws/aws-sdk-go-v2/service/s3 v1.63.3 + github.com/aws/aws-sdk-go-v2 v1.31.0 + github.com/aws/aws-sdk-go-v2/config v1.27.39 + github.com/aws/aws-sdk-go-v2/credentials v1.17.37 + github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.25 + github.com/aws/aws-sdk-go-v2/service/s3 v1.63.3
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (1)
go.mod(12 hunks)
🔇 Additional comments (2)
go.mod (2)
Line range hint 415-416: LGTM: Retraction notice is well documented.
The retraction notice for v3.2.0 is properly documented with a clear reason.
71-71: Verify SOPS integration dependencies.
The addition of SOPS-related dependencies (github.com/getsops/sops/v3 and github.com/getsops/gopgagent) aligns with the PR objective of integrating SOPS for encrypted secret files. However, let's verify if all necessary cloud provider SDKs are included for comprehensive key management support.
Also applies to: 179-179
✅ Verification successful
All required cloud provider SDKs for SOPS integration are present
The verification confirms that the necessary cloud provider SDKs for SOPS key management are properly included in the dependencies:
- AWS KMS:
github.com/aws/aws-sdk-go-v2/service/kms - Google Cloud KMS:
cloud.google.com/go/kms - Azure Key Vault:
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check if all common SOPS-related cloud provider SDKs are present
# Expected: Should find dependencies for major cloud providers (AWS KMS, GCP KMS, Azure Key Vault)
echo "Checking SOPS-related dependencies..."
rg "github.com/aws/aws-sdk-go-v2/service/kms|cloud.google.com/go/kms|azure.*keyvault" go.mod
Length of output: 454
Proposed changes
Close #5840
Note
github.com/getsops/sops/v3requires go >= 1.22.How has this been tested?
1. Build
2. Test
3. Proof
With encrypted values:
Regular secret file (non-encrypted):
4. Cleanup
Checklist
Summary by CodeRabbit
Release Notes
New Features
.sops.yaml.example-auth.enc.yaml, containing static and dynamic secrets.Documentation
Chores
go.mod..gitignoreto track.sops.yamland.golangci.yamlfiles..golangci.yamlfor excluding specific linting rules.