Scan failures (java)

Sonatype Lift
1

I have a large java based project which supports build in maven & gradle (we are moving from one to the other).

I noticed scans are failing. The first issue was due to java11, to I have set that in the .lift.toml. Still remaining I have these observations:

A) ErrorProne

This is failing with entries like:

Running V2 API tools

Running Error Prone

Capture:

Failed when invoking process

Error Prone complete in 15.526095292s

ErrorProne_JDK11 tool error: Capture:

Execution Failed: Script "javac -J-classpath -J/opt/errorprone-jars/error_prone_core-2.3.3-with-dependencies.jar:/opt/errorprone-jars/dataflow-2.5.7.jar:/opt/errorprone-jars/javacutil-2.5.7.jar:/opt/errorprone-jars/jFormatString-3.0.0.jar:/opt/errorprone-jars/guava.jar -XDcompilePolicy=simple -processorpath /opt/errorprone-jars/error_prone_core-2.3.3-with-dependencies.jar:/opt/errorprone-jars/dataflow-2.5.7.jar:/opt/errorprone-jars/javacutil-2.5.7.jar:/opt/errorprone-jars/jFormatString-3.0.0.jar -Xplugin:ErrorProne -XepAllErrorsAsWarnings -XepDisableAllChecks-Xep:AndroidInjectionBeforeSuper:WARN-Xep:ArrayEquals:WARN-Xep:ArrayFillIncompatibleType:WARN-Xep:ArrayHashCode:WARN-Xep:ArrayToString:WARN-Xep:ArraysAsListPrimitiveArray:WARN-Xep:AsyncCallableReturnsNull:WARN-Xep:AsyncFunctionReturnsNull:WARN-Xep:AutoValueConstructorOrderChecker:WARN-Xep:BadShiftAmount:WARN-Xep:BundleDeserializationCast:WARN-Xep:ChainingConstructorIgnoresParameter:WARN-Xep:CheckReturnValue:WARN-Xep:CollectionIncompatibleType:WARN-Xep:CollectionToArraySafeParameter:WARN-Xep:ComparableType:WARN-Xep:ComparingThisWithNull:WARN-Xep:ComparisonOutOfRange:WARN-Xep:CompatibleWithAnnotationMisuse:WARN-Xep:ConditionalExpressionNumericPromotion:WARN-Xep:ConstantOverflow:WARN-Xep:DaggerProvidesNull:WARN-Xep:DeadException:WARN-Xep:DeadThread:WARN-Xep:DiscardedPostfixExpression:WARN-Xep:DoNotCall:WARN-Xep:DuplicateMapKeys:WARN-Xep:DurationFrom:WARN-Xep:DurationGetTemporalUnit:WARN-Xep:DurationToLongTimeUnit:WARN-Xep:EqualsHashCode:WARN-Xep:EqualsNaN:WARN-Xep:EqualsReference:WARN-Xep:EqualsWrongThing:WARN-Xep:ForOverride:WARN-Xep:FormatString:WARN-Xep:FormatStringAnnotation:WARN-Xep:FunctionalInterfaceMethodChanged:WARN-Xep:FuturesGetCheckedIllegalExceptionType:WARN-Xep:GetClassOnAnnotation:WARN-Xep:GetClassOnClass:WARN-Xep:GuardedBy:WARN-Xep:GuiceAssistedInjectScoping:WARN-Xep:GuiceAssistedParameters:WARN-Xep:GuiceInjectOnFinalField:WARN-Xep:HashtableContains:WARN-Xep:IdentityBinaryExpression:WARN-Xep:Immutable:WARN-Xep:ImmutableModification:WARN-Xep:IncompatibleArgumentType:WARN-Xep:IndexOfChar:WARN-Xep:InexactVarargsConditional:WARN-Xep:InfiniteRecursion:WARN-Xep:InjectMoreThanOneScopeAnnotationOnClass:WARN-Xep:InjectOnMemberAndConstructor:WARN-Xep:InvalidPatternSyntax:WARN-Xep:InvalidTimeZoneID:WARN-Xep:InvalidZoneId:WARN-Xep:IsInstanceOfClass:WARN-Xep:IsLoggableTagLength:WARN-Xep:JUnit3TestNotRun:WARN-Xep:JUnit4ClassAnnotationNonStatic:WARN-Xep:JUnit4SetUpNotRun:WARN-Xep:JUnit4TearDownNotRun:WARN-Xep:JUnit4TestNotRun:WARN-Xep:JUnitAssertSameCheck:WARN-Xep:JavaxInjectOnAbstractMethod:WARN-Xep:JodaToSelf:WARN-Xep:LiteByteStringUtf8:WARN-Xep:LoopConditionChecker:WARN-Xep:MathRoundIntLong:WARN-Xep:MislabeledAndroidString:WARN-Xep:MissingSuperCall:WARN-Xep:MissingTestCall:WARN-Xep:MisusedWeekYear:WARN-Xep:MockitoCast:WARN-Xep:MockitoUsage:WARN-Xep:ModifyingCollectionWithItself:WARN-Xep:MoreThanOneInjectableConstructor:WARN-Xep:MustBeClosedChecker:WARN-Xep:NCopiesOfChar:WARN-Xep:NonCanonicalStaticImport:WARN-Xep:NonFinalCompileTimeConstant:WARN-Xep:NonRuntimeAnnotation:WARN-Xep:NullTernary:WARN-Xep:OptionalEquality:WARN-Xep:OverlappingQualifierAndScopeAnnotation:WARN-Xep:OverridesJavaxInjectableMethod:WARN-Xep:PackageInfo:WARN-Xep:ParcelableCreator:WARN-Xep:PeriodFrom:WARN-Xep:PeriodGetTemporalUnit:WARN-Xep:PeriodTimeMath:WARN-Xep:PreconditionsCheckNotNull:WARN-Xep:PreconditionsCheckNotNullPrimitive:WARN-Xep:PredicateIncompatibleType:WARN-Xep:PrivateSecurityContractProtoAccess:WARN-Xep:ProtoFieldNullComparison:WARN-Xep:ProtoStringFieldReferenceEquality:WARN-Xep:ProtoTruthMixedDescriptors:WARN-Xep:ProtocolBufferOrdinal:WARN-Xep:ProvidesMethodOutsideOfModule:WARN-Xep:RandomCast:WARN-Xep:RandomModInteger:WARN-Xep:RectIntersectReturnValueIgnored:WARN-Xep:RefersToDaggerCodegen:WARN-Xep:ReturnValueIgnored:WARN-Xep:SelfAssignment:WARN-Xep:SelfComparison:WARN-Xep:SelfEquals:WARN-Xep:ShouldHaveEvenArgs:WARN-Xep:SizeGreaterThanOrEqualsZero:WARN-Xep:StreamToString:WARN-Xep:StringBuilderInitWithChar:WARN-Xep:SubstringOfZero:WARN-Xep:SuppressWarningsDeprecated:WARN-Xep:TemporalAccessorGetChronoField:WARN-Xep:ThrowIfUncheckedKnownChecked:WARN-Xep:ThrowNull:WARN-Xep:TruthSelfEquals:WARN-Xep:TryFailThrowable:WARN-Xep:TypeParameterQualifier:WARN-Xep:UnnecessaryTypeArgument:WARN-Xep:UnusedAnonymousClass:WARN-Xep:UnusedCollectionModifiedInPlace:WARN-Xep:VarTypeName:WARN -cp /tmp/analyzing-504ed5fdc12649a8/open-metadata-test/open-metadata-ut/target/classes:/root/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.13.0/jackson-annotations-2.13.0.jar:/tmp/analyzing-504ed5fdc12649a8/open-metadata-implementation/frameworks/audit-log-framework/target/audit-log-framework-3.3-SNAPSHOT.jar:/root/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.13.0/jackson-core-2.13.0.jar:/root/.m2/repository/org/slf4j/slf4j-api/1.7.32/slf4j-api-1.7.32.jar:/tmp/analyzing-504ed5fdc12649a8/open-metadata-implementation/frameworks/open-connector-framework/target/open-connector-framework-3.3-SNAPSHOT.jar:/root/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.13.0/jackson-databind-2.13.0.jar:/root/.m2/repository/com/beust/jcommander/1.78/jcommander-1.78.jar:/root/.m2/repository/org/testng/testng/7.4.0/testng-7.4.0.jar:/root/.m2/repository/org/webjars/jquery/3.5.1/jquery-3.5.1.jar:/root/.m2/repository/org/slf4j/slf4j-simple/1.7.32/slf4j-simple-1.7.32.jar -d /tmp/analyzing-504ed5fdc12649a8/open-metadata-test/open-metadata-ut/target/classes --release 11 /tmp/analyzing-504ed5fdc12649a8/open-metadata-test/open-metadata-ut/src/main/java/org/odpi/openmetadata/test/unittest/utilities/BeanTestBase.java /tmp/analyzing-504ed5fdc12649a8/open-metadata-test/open-metadata-ut/src/main/java/org/odpi/openmetadata/test/unittest/utilities/ExceptionMessageSetTest.java /tmp/analyzing-504ed5fdc12649a8/open-metadata-test/open-metadata-ut/src/main/java/org/odpi/openmetadata/test/unittest/utilities/MessageSetTest.java /tmp/analyzing-504ed5fdc12649a8/open-metadata-test/open-metadata-ut/src/main/java/org/odpi/openmetadata/test/unittest/utilities/OCFCheckedExceptionBasedTest.java /tmp/analyzing-504ed5fdc12649a8/open-metadata-test/open-metadata-ut/src/main/java/org/odpi/openmetadata/test/unittest/utilities/AuditLogMessageSetTest.java" exited with code 4.

This is a rather cumbersome command line. I’ve not used error-prone much manually - we could add a profile to maven, but haven’t so far. Also worth noting we do use lombok in a few areas -

B) findsecbugs

I’ve run this tool before, but when lift invokes it I see lots of entries like


Running V3 API (build-based) tools
Running FindSecBugs
Standard error: 
Called: ['/opt/findsecbugs/findsecbugs.sh', '-low', '-xml:withMessages', '-output', './findsecbugs-out/report.xml', '/tmp/analyzing-504ed5fdc12649a8/open-metadata-implementation/frameworks/audit-log-framework/target/classes']
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#noProviders for further details.

Now I am very aware of the requirement for SLF4J bindings - ie one and only one. I am unsure though if this will affect the scan or not, and if it does how to fix

C) OSS Vulnarabilities

We get the output of this scan when our artifacts are released via oss.sonatype.org (for maven central publishing), but within our triggered builds I just see

Running V1 API (bulk) tools

Running Open Source Vulnerabilities

Sanitizing repository

Analysis failed in 2m49s

it’s not clear WHY this failed…

Any tips?

2

@nigel.l.jones Thank you for filing, I’ve asked the engineering teams to take a look. We have different teams looking into these for you so responses to each point may come through at different times.

To confirm, the project you’re analyzing is https://lift.sonatype.com/odpi/egeria?

3

Yes - that is the project, sourced from https://github.com/odpi/egeria
(We have some smaller repos that we’re also hoping to use lift on, with a few in-place, but I think if we understand the main project the others will be fine!)

1 Like
4

@nigel.l.jones we have taken a look at your issue, in particular the OSS Vulnerabilities issue) and have a fix going through. Hopefully, the fix will be released in the next week or so. I will update you when this has been done.

Thanks for being patient

2 Likes
5

Thanks - this week I opened up a thread on PR annotations, but I subsequently added more
references to the build itself (which then failed). See Pull request annotations - #8 by nigel.l.jones

6

Hi Nigel,
Can you confirm you are or are not experiencing B above still?
I had forked Egeria earlier this week to test on your other threads and did not see this happening on my fork.
Coincidentally, I came across the ticket in our tracking system today and thought I’d double check. Either it’s intermittant or just happening for you and not me (latter would be weird, but not unheard of).
Thanks,
Joe

7

Hi,
I checked the build logs from sonatype on a recent PR ( Sonatype Lift -- Console ) and noticed

  • The bill of materials tool fails with running out of Heap (:16543)

I didn’t notice an error from findsecbugs - and the other tools seemed to work.

I did note that viewing the build logs is a bit tricky when they are large - the find option does a filter, but then hard to see context. A download would be most useful. The log for our main project is nearly 37000 lines long

8

I should add that I do have a new v4 branch which is failing, but that’s my work in progress (we’re shifting from a default of java 11/maven to 17/gradle and part way through changing the github actions & updating .toml etc)

9

Thanks for the report back.

I relayed the BoM heap error to the appropriate team. It is not familiar to me but they may reply for more information.

I agree with you on the logs and believe we have an internal enhancement about it. I’ll add your comment to that ticket.

I assume if you need someone to look at the v4 branch, you’ll reach out. Good luck with the migration!