I have Sonatype Lift setup on our repository github com odpi egeria
I noticed a PR github com odpi egeria pull 6030 reported ’ lift — Complete (154 min, 10/10 checks) 107 new bugs found’
Yet only 4 annotations have been made on the PR
What should I expect? Is it capped at a max number, or do only some issues get annotated? (these are all NEW)
I checked some other PRs. Another 6029, reports
’ lift — Complete (160 min, 10/10 checks) 48 new bugs found’
yet there are no annotations in the PR
What I’m trying to figure out is how to ‘encourage’ more of these issues to be addressed during the PR validation process rather than ignored and be fixed later.
First step was to look at the PR annotations as it makes the issues more visible (and constructive).
Nigel,
Thank you for bringing this up, I think I can clear up the situation and the good news is it isn’t that there are hundreds of bugs being introduced by your contributors. Some of the tools Lift has integrated - most notably Errorprone and Infer - have edge cases where they behave non-deterministically. This means bugs will be reported some times and not others, thus falsely appearing as fixed or new. Your project seems to exhibit this behavior in abundance and will make a good test case.
Right now Lift will comment on all newly detected bugs that are also in the PR’s diff - at or near lines which have changed. If Lift does not produce a PR comment then the chances are good that the issue is not relevant to the PR.
We’re also considering how to better present the information as well as reduce the noise caused by the nondeterministic operation so you and others will be presented with a clearer picture from the beginning.
EDIT: I’ve some poor phrasing that the tool behaves non-deterministically. I think errorprone is consistent but we know its integration is not. Much appreciation to the errorprone team and other maintainers.
Thanks for the clarification - that’s good to know.
I also noticed the scan takes an age (unfortunately a big project). We’re running at around 160 mins for a scan. Regular github maven build is around 35-40 mins, we also build under gradle which is down at the 12mins. (the project contains both pom.xml and build.gradle so my first step as per Configuring Lift is likely to force gradle.
Yes, specifying build = "gradlew" is probably the easiest step to take. I see the build is currently using Maven so it will be educational for me to find out if the misbehavior is impacted by using gradle instead.
I tried the change to gradle, with one tweak:
# SPDX-License-Identifier: Apache-2.0
# Copyright Contributors to the ODPi Egeria project.
jdk11 = true
build = "gradlew -x test -x javadoc build"
I added the two skips for test/javadoc in order to improve the gradle build time - both steps take qutie a while, in particular test as we run some functional tests which involve starting up our applications fully
With this done the build doesn’t seem to work - I get ‘Analysis Failed’ at Sonatype Lift -- Console
It does seem as if lift is trying gradle ie:
Detecting the build system based on configurations and files present in the repository.
Using build systems of [([],Gradlew ["-x","test","-x","javadoc","build"])] to build compilation database.
Running Gradlew
And after lots of files being listed (5k)
Compilation database generation succeeded using Gradlew ["-x","test","-x","javadoc","build"] and resulted in a database of length 278
findsecbugs runs and reports lots of logging errors (there is no binding)
Standard error:
Called: ['/opt/findsecbugs/findsecbugs.sh', '-low', '-xml:withMessages', '-output', './findsecbugs-out/report.xml', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-test/open-metadata-fvt/access-services-fvt/subject-area-fvt/build/classes/java/main']
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#noProviders for further details.
finally ending in
Found 0 bugs in 18m28s
which seems surprising … 0 bugs? really? I doubt it …
Errorprone then finds a fair few:
Running ErrorProne
Found 4568 bugs in 48m47s
Infer then runs, though no idea what it finds:
Running Infer
Standard error:
Called: ['infer', 'capture', '-o', 'infer-out', '--continue', '-r', '--keep-going', '--', 'javac', '-cp', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/build/classes/java/main:/home/lift/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/home/lift/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.2/a770cc4c0a1fb0bfd8a150a6a0004e42bc99fca/jackson-annotations-2.12.2.jar', '-d', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/build/classes/java/main', '-cp', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/build/classes/java/main:/home/lift/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/home/lift/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.2/a770cc4c0a1fb0bfd8a150a6a0004e42bc99fca/jackson-annotations-2.12.2.jar:/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/build/classes/java/main', '-d', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/build/classes/java/main', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/AuditLog.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/AuditLogDestination.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/AuditLogRecord.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/AuditLogReport.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/AuditLogReportingComponent.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/AuditLoggingComponent.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/ComponentDescription.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/MessageFormatter.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/messagesets/AuditLogMessageDefinition.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/messagesets/AuditLogMessageSet.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/messagesets/AuditLogRecordSeverity.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/messagesets/ExceptionMessageDefinition.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/messagesets/ExceptionMessageSet.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/src/main/java/org/odpi/openmetadata/frameworks/auditlog/messagesets/MessageDefinition.java']
Capturing in javac mode...
Standard error:
Called: ['infer', 'capture', '-o', 'infer-out', '--continue', '-r', '--keep-going', '--', 'javac', '-cp', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/build/classes/java/main:/home/lift/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/home/lift/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.2/a770cc4c0a1fb0bfd8a150a6a0004e42bc99fca/jackson-annotations-2.12.2.jar:/home/lift/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.12.2/8df50138521d05561a308ec2799cc8dda20c06df/jackson-core-2.12.2.jar:/home/lift/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.12.2/5f9d79e09ebf5d54a46e9f4543924cf7ae7654e0/jackson-databind-2.12.2.jar:/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/build/libs/audit-log-framework-3.5-SNAPSHOT.jar', '-d', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/build/classes/java/main', '-cp', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/build/classes/java/main:/home/lift/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/home/lift/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.2/a770cc4c0a1fb0bfd8a150a6a0004e42bc99fca/jackson-annotations-2.12.2.jar:/home/lift/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.12.2/8df50138521d05561a308ec2799cc8dda20c06df/jackson-core-2.12.2.jar:/home/lift/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.12.2/5f9d79e09ebf5d54a46e9f4543924cf7ae7654e0/jackson-databind-2.12.2.jar:/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/audit-log-framework/build/libs/audit-log-framework-3.5-SNAPSHOT.jar:/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/build/classes/java/main', '-d', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/build/classes/java/main', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/Connector.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ConnectorBase.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ConnectorBroker.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ConnectorProvider.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ConnectorProviderBase.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/SampleDataExtension.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/VirtualConnectorExtension.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/ConnectionCheckedException.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/ConnectorCheckedException.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/InvalidParameterException.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/OCFCheckedExceptionBase.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/OCFErrorCode.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/OCFRuntimeException.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/PropertyServerException.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/UserNotAuthorizedException.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/ffdc/package-info.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AdditionalProperties.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetAPIOperation.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetAPIOperations.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetAPISchemaType.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetCertification.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetCertifications.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetClassification.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetComment.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetCommentReplies.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetComments.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetComplexSchemaType.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetConnections.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetDescriptor.java', '/tmp/analyzing-9409a9d0cf44da1e/open-metadata-implementation/frameworks/open-connector-framework/src/main/java/org/odpi/openmetadata/frameworks/connectors/properties/AssetDetail.java........ [ removed text as too long .. this is to give an idea of what is in the log ...... ]
Capturing in javac mode...
Warning: Debug information of local_variable_table attribute of method org.odpi.openmetadata.frameworks.connectors.Connector org.odpi.openmetadata.frameworks.connectors.ConnectorBroker.getConnector(org.odpi.openmetadata.frameworks.connectors.properties.ConnectionProperties) cannot be used for code transformation because it is inconsistent on localvar 6 at program point 141.
Warning: Debug information of local_variable_table attribute of method org.odpi.openmetadata.frameworks.connectors.ConnectorProvider org.odpi.openmetadata.frameworks.connectors.ConnectorBroker.getConnectorProvider(org.odpi.openmetadata.frameworks.connectors.properties.ConnectorTypeProperties, java.lang.String, java.lang.String) cannot be used for code transformation because it is inconsistent on localvar 5 at program point 114.
Warning: Debug information of local_variable_table attribute of method org.odpi.openmetadata.frameworks.connectors.Connector org.odpi.openmetadata.frameworks.connectors.ConnectorProviderBase.getConnector(org.odpi.openmetadata.frameworks.connectors.properties.ConnectionProperties) cannot be used for code transformation because it is inconsistent on localvar 3 at program point 174.
though I note a few of those compile errors
I then get the same again - is this the case of running on the base commit, then with the PR? hence double build time?
Overall though this process takes a long time – say 4x build time - we’re into 2-4hours here rather than the gradle time of say 15mins
So I’m trying to figure out how we setup ‘good’ scanning of this repo - in terms of build setup, balancing with performance, and accuracy
The goal is to see if a scan can come down to 45-50 mins which could mean we make the check mandatory before merge?
The summary (inserted into the ‘check’ report on the PR) reports ’ Internal issue, but the bot won’t block the build (we’re working on a fix).’