Find duplicate source files

org.openrewrite.FindDuplicateSourceFiles

Record the presence of LSTs with duplicate paths, indicating that the same file was parsed more than once.

Recipe source

GitHub: FindDuplicateSourceFiles.java, Issue Tracker, Code Genome Project

This recipe is available under the Apache License Version 2.0.

Example

hello.properties

Before

hello.properties

hello=world

After

hello.properties

~~>hello=world

Diff

--- hello.properties
+++ hello.properties
@@ -1,1 +1,1 @@
-hello=world
+~~>hello=world

Usage

This recipe has no required configuration options. It can be activated by adding a dependency on `org.openrewrite.recipe:rewrite-all` in your build file or by running a shell command (in which case no build changes are needed):

OpenRewrite artifacts are distributed through the Code Genome Project repository (https://artifacts.codegenomeproject.org/maven), which requires authentication. Sign in to the Code Genome Project to create a download token, then in the snippets below replace USERNAME with the email or username you signed in with and TOKEN with that token. See the quickstart guide for details.

Gradle

1. Add the following to your build.gradle file:
   build.gradle

   plugins {
       id("org.openrewrite.rewrite") version("latest.release")
   }
   
   rewrite {
       activeRecipe("org.openrewrite.FindDuplicateSourceFiles")
       setExportDatatables(true)
   }
   
   repositories {
       mavenCentral()
       maven {
           url = "https://artifacts.codegenomeproject.org/maven"
           credentials {
               username = "USERNAME"
               password = "TOKEN"
           }
       }
   }
   
   dependencies {
       rewrite("org.openrewrite.recipe:rewrite-all:1.28.1")
   }

2. Run gradle rewriteRun to run the recipe.

Gradle init script

1. Create a file named init.gradle in the root of your project.
   init.gradle

   initscript {
       repositories {
           maven { url "https://plugins.gradle.org/m2" }
       }
       dependencies { classpath("org.openrewrite:plugin:7.39.0") }
   }
   rootProject {
       plugins.apply(org.openrewrite.gradle.RewritePlugin)
       dependencies {
           rewrite("org.openrewrite.recipe:rewrite-all:1.28.1")
       }
       rewrite {
           activeRecipe("org.openrewrite.FindDuplicateSourceFiles")
       setExportDatatables(true)
       }
       afterEvaluate {
           if (repositories.isEmpty()) {
               repositories {
                   mavenCentral()
               }
           }
           repositories {
               maven {
                   url = "https://artifacts.codegenomeproject.org/maven"
                   credentials {
                       username = "USERNAME"
                       password = "TOKEN"
                   }
               }
           }
       }
   }

2. Run the recipe.
   shell

   gradle --init-script init.gradle rewriteRun

Gradle init script (Kotlin)

1. Create a file named init.gradle.kts in the root of your project.
   init.gradle.kts

   initscript {
       repositories {
           maven { url = uri("https://plugins.gradle.org/m2") }
       }
       dependencies { classpath("org.openrewrite:plugin:7.39.0") }
   }
   rootProject {
       plugins.apply(org.openrewrite.gradle.RewritePlugin::class.java)
       dependencies {
           add("rewrite", "org.openrewrite.recipe:rewrite-all:1.28.1")
       }
       extensions.configure<org.openrewrite.gradle.RewriteExtension> {
           activeRecipe("org.openrewrite.FindDuplicateSourceFiles")
       setExportDatatables(true)
       }
       afterEvaluate {
           if (repositories.isEmpty()) {
               repositories {
                   mavenCentral()
               }
           }
           repositories {
               maven {
                   url = uri("https://artifacts.codegenomeproject.org/maven")
                   credentials {
                       username = "USERNAME"
                       password = "TOKEN"
                   }
               }
           }
       }
   }

2. Run the recipe.
   shell

   gradle --init-script init.gradle.kts rewriteRun

Maven POM

1. Add the Code Genome Project credentials to your Maven settings.xml file (typically at ~/.m2/settings.xml):
   settings.xml

   <settings>
     <servers>
       <server>
         <id>codegenome</id>
         <username>USERNAME</username>
         <password>TOKEN</password>
       </server>
     </servers>
   </settings>

2. Add the following to your pom.xml file:
   pom.xml

   <project>
     <repositories>
       <repository>
         <id>codegenome</id>
         <url>https://artifacts.codegenomeproject.org/maven</url>
       </repository>
     </repositories>
     <pluginRepositories>
       <pluginRepository>
         <id>codegenome</id>
         <url>https://artifacts.codegenomeproject.org/maven</url>
       </pluginRepository>
     </pluginRepositories>
     <build>
       <plugins>
         <plugin>
           <groupId>org.openrewrite.maven</groupId>
           <artifactId>rewrite-maven-plugin</artifactId>
           <version>6.46.1</version>
           <configuration>
             <exportDatatables>true</exportDatatables>
             <activeRecipes>
               <recipe>org.openrewrite.FindDuplicateSourceFiles</recipe>
             </activeRecipes>
           </configuration>
           <dependencies>
             <dependency>
               <groupId>org.openrewrite.recipe</groupId>
               <artifactId>rewrite-all</artifactId>
               <version>1.28.1</version>
             </dependency>
           </dependencies>
         </plugin>
       </plugins>
     </build>
   </project>

3. Run mvn rewrite:run to run the recipe.

Maven Command Line

You will need to have Maven installed on your machine before you can run the following command. Since this approach leaves your pom.xml untouched, the Code Genome Project repository has to be declared in your Maven settings.xml file (typically at ~/.m2/settings.xml) instead: as a pluginRepository so that rewrite-maven-plugin itself can be resolved, and as a repository so that recipe artifacts passed to rewrite.recipeArtifactCoordinates can be resolved.

settings.xml

<settings>
  <servers>
    <server>
      <id>codegenome</id>
      <username>USERNAME</username>
      <password>TOKEN</password>
    </server>
  </servers>
  <profiles>
    <profile>
      <id>codegenome</id>
      <repositories>
        <repository>
          <id>codegenome</id>
          <url>https://artifacts.codegenomeproject.org/maven</url>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>codegenome</id>
          <url>https://artifacts.codegenomeproject.org/maven</url>
        </pluginRepository>
      </pluginRepositories>
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>codegenome</activeProfile>
  </activeProfiles>
</settings>

With those credentials in place you can run the recipe:

shell

mvn -U org.openrewrite.maven:rewrite-maven-plugin:6.46.1:run --define rewrite.recipeArtifactCoordinates=org.openrewrite.recipe:rewrite-all:RELEASE --define rewrite.activeRecipes=org.openrewrite.FindDuplicateSourceFiles --define rewrite.exportDatatables=true

Moderne CLI

You will need to have configured the Moderne CLI on your machine before you can run the following command.

shell

mod run . --recipe FindDuplicateSourceFiles

If the recipe is not available locally, then you can install it using:

mod config recipes jar install org.openrewrite.recipe:rewrite-all:1.28.1

See how this recipe works across multiple open-source repositories

Run this recipe on OSS repos at scale with the Moderne SaaS.

Run this recipe here

The community edition of the Moderne platform enables you to easily run recipes across thousands of open-source repositories.

Please contact Moderne for more information about safely running the recipes on your own codebase in a private SaaS.

Data Tables

DuplicateSourceFiles

Duplicate source files

org.openrewrite.table.DuplicateSourceFiles

A list of source files that occur more than once in an LST.

Column Name	Description
Source path	The path to the source file.
Count	The number of times an LST element with this source file path is present.
Types	The LST types of the source file.

SourcesFileResults

Source files that had results

org.openrewrite.table.SourcesFileResults

Source files that were modified by the recipe run.

Column Name	Description
Source path before the run	The source path of the file before the run. null when a source file was created during the run.
Source path after the run	A recipe may modify the source path. This is the path after the run. null when a source file was deleted during the run.
Parent of the recipe that made changes	In a hierarchical recipe, the parent of the recipe that made a change. Empty if this is the root of a hierarchy or if the recipe is not hierarchical at all.
Recipe that made changes	The specific recipe that made a change.
Estimated time saving	An estimated effort that a developer to fix manually instead of using this recipe, in unit of seconds.
Cycle	The recipe cycle in which the change was made.

SearchResults

Source files that had search results

org.openrewrite.table.SearchResults

Search results that were found during the recipe run.

Column Name	Description
Source path of search result before the run	The source path of the file with the search result markers present.
Source path of search result after run the run	A recipe may modify the source path. This is the path after the run. null when a source file was deleted during the run.
Result	The trimmed printed tree of the LST element that the marker is attached to.
Description	The content of the description of the marker.
Recipe that added the search marker	The specific recipe that added the Search marker.

SourcesFileErrors

Source files that errored on a recipe

org.openrewrite.table.SourcesFileErrors

The details of all errors produced by a recipe run.

Column Name	Description
Source path	The file that failed to parse.
Recipe that made changes	The specific recipe that made a change.
Stack trace	The stack trace of the failure.

RecipeRunStats

Recipe performance

org.openrewrite.table.RecipeRunStats

Statistics used in analyzing the performance of recipes.

Column Name	Description
The recipe	The recipe whose stats are being measured both individually and cumulatively.
Source file count	The number of source files the recipe ran over.
Source file changed count	The number of source files which were changed in the recipe run. Includes files created, deleted, and edited.
Cumulative scanning time (ns)	The total time spent across the scanning phase of this recipe.
Max scanning time (ns)	The max time scanning any one source file.
Cumulative edit time (ns)	The total time spent across the editing phase of this recipe.
Max edit time (ns)	The max time editing any one source file.