How to trigger build steps based on modified directory?

"Life is all about sharing. If we are good at something, pass it on." - Mary Berry

How to trigger build steps based on modified directory?

2020-06-26

Categories: DevOps

Using monorepo with multiple micro services, every single commit will trigger a full lint/test/build/publish for every service. What can I do to limit the scope?

To do that, we can use git diff to show changes between commits:

 1  if [[ -n "${DRONE_PULL_REQUEST}" ]]; then
 2    most_recent_before="origin/${DRONE_TARGET_BRANCH}"
 3  elif [[ "${DRONE_BUILD_EVENT}" = "push" && ("${DRONE_COMMIT_BRANCH}" = "master" || "${DRONE_COMMIT_BRANCH}" = "release-"*) ]]; then
 4    if [[ "${DRONE_COMMIT_BEFORE}" = "$zero" ]]; then
 5      exit 0
 6    else
 7      most_recent_before="${DRONE_COMMIT_BEFORE}"
 8    fi
 9  fi
10  modified_files=$(git --no-pager diff --name-only "${DRONE_COMMIT_SHA}".."${most_recent_before}");

Moreover, if go.mod is changed, all affected modules should be re-tested:

1  if echo "$modified_files" | grep -q "go.mod"; then
2    modified_deps=$(git diff --unified=0 "${DRONE_COMMIT_SHA}".."${most_recent_before}" go.mod | tail +6 | grep '^[+-]' | awk '{ print $2 }' | uniq)
3    while read -r module; do
4      if go list -f '{{ join .Deps "\n" }}' "./${module}" | grep -qf <(echo "${modified_deps}"); then
5        echo "$module"
6      fi
7    done < <(echo "$all_modules_in_cluster")
8  fi

Run this script, we will have a list of modified services:

1local listModifiedModules(cluster) = {
2    name: "list-modified-modules",
3    image: golang,
4    volumes: mounts,
5	commands: [
6	    "./scripts/show_modified_folders.sh " + cluster + " > " + cluster + "/modified_modules.txt"
7	],
8};

Then the lint/test/build step can be run on that list:

 1local lint(cluster) = {
 2	name: "lint",
 3	image: golangci,
 4	volumes: mounts,
 5	environment: {
 6	    "GOPACKAGESPRINTGOLISTERRORS": 1,
 7	},
 8	commands: [
 9		"./scripts/linter.sh " + cluster + "/modified_modules.txt"
10	],
11	depends_on: [
12		"list-modified-modules"
13	],
14};

The last step we need to do is publish only changed services by using drone-convert-pathschanged conversion extension:

 1local publishPush(cluster, module) = {
 2	name: "publish-" + module,
 3	image: docker,
 4	settings: {
 5	},
 6	depends_on: [
 7		"build"
 8	],
 9	when: {
10		branch: [
11			"master",
12			"release-*"
13		],
14		event: [
15		    "push",
16		],
17		paths: {
18		    include: [
19		        cluster + "/" + module + "/**/*.go"
20		    ],
21		},
22	}
23};

Tags: drone ci monorepo

Edit on GitHub