DEV Community

Cover image for Consume artifacts from a remote DevOps project pipeline
Marcel.L
Marcel.L

Posted on • Updated on

Consume artifacts from a remote DevOps project pipeline

Azure DevOps pipeline/YAML resources

Azure DevOps pipelines provides very useful resources we can define in our pipeline in one place and be consumed anywhere in our pipeline.

A resource is anything used by a pipeline that lives outside the pipeline. Pipeline resources include:

  • CI/CD pipelines that produce artifacts (Azure Pipelines, Jenkins, etc.)
  • code repositories (Azure Repos Git repos, GitHub, GitHub Enterprise, Bitbucket Cloud)
  • container image registries (Azure Container Registry, Docker Hub, etc.)
  • package feeds (GitHub packages)

Today we will take a look at the Pipelines Resource, in particular we will look at how we can use this resource in a pipeline to consume an artifact that was produced in another pipeline in a completely different project. Our pipeline will also even be triggered automatically by the source pipeline after the artifact has been created and published.

Consume remote pipeline artifacts

In my DevOps organisation I have created two projects namely ProjectA and ProjectB. I also created two YAML pipelines for each corresponding project named PipelineA and PipelineB. PipelineA will be my triggering/source pipeline which will create an artifact called ArtifactA. PipelineB will be my pipeline which will contain the pipeline resource for PipelineA and will consume ArtifactA.

main

In ProjectA I also created a repository called RepoA which contains a file called MyConfig.txt.

projects

myConfig

I also created the following code in PipelineA.yml.

## code/PipelineA.yml

trigger: none

stages:
- stage: Build_Artifact
  displayName: Build Artifact A

  jobs:
  - job: Build
    displayName: Build
    pool:
      name: Azure Pipelines
      vmImage: windows-2019

    steps:
    - task: CopyFiles@2
      displayName: 'Copy myConfig to Staging'
      inputs:
        SourceFolder: '$(Build.SourcesDirectory)'
        Contents: 'MyConfig.txt'
        TargetFolder: '$(Build.ArtifactStagingDirectory)/drop'

    - task: PublishPipelineArtifact@1
      displayName: 'Publish Artifact to Pipeline'
      inputs:
        targetPath: '$(Build.ArtifactStagingDirectory)/drop'
        artifactName: ArtifactA
Enter fullscreen mode Exit fullscreen mode

NOTE: It is important to note that when we create the above pipeline in our source project we must rename the pipeline to the same name as what we will refer to it in our pipeline resource on PipelineB. In my case I will refer to this as PipelineA.

rename

The above YAML pipeline will take the file MyConfig.txt and create a pipeline artifact containing the file called ArtifactA.

pipelineA

artifactA

In ProjectB I have PipelineB.yml that contains the pipeline resource for PipelineA and will be triggered once PipelineA completes and we will use the download task to also consume the artifact that was produced by PipelineA.

## code/PipelineB.yml

trigger: none
pr: none

# ------ This is our Pipeline Resource ------
resources:
  pipelines:
  - pipeline: PipelineA  # identifier for the resource used in pipeline resource variables.
    project: ProjectA    # project for the source; optional for current project.
    source: PipelineA    # name of the pipeline that produces an artifact.
    trigger:             # triggers are not enabled by default unless you add trigger section to the resource.
      branches:          # branch conditions to filter the events, optional; Defaults to all branches.
        include:         # branches to consider the trigger events, optional; Defaults to all branches.
        - main
# ------------------------------------------

stages:
- stage: Consume_Artifact
  displayName: Consume Artifact A

  jobs:
  - job: Consume
    displayName: Consume
    pool:
      name: Azure Pipelines
      vmImage: windows-2019

    steps:
    - task: PowerShell@2
      displayName: 'Information'
      inputs:
        targetType: inline
        script: |
          Write-output "This pipeline has been triggered by: $(resources.pipeline.PipelineA.pipelineName)"

    - download: PipelineA
      artifact: 'ArtifactA'

    - task: PowerShell@2
      displayName: 'Get-Content MyConfig.txt'
      inputs:
        targetType: inline
        script: |
          Get-Content -path $(Pipeline.Workspace)/PipelineA/ArtifactA/MyConfig.txt
Enter fullscreen mode Exit fullscreen mode

NOTE: It is important to note that we have to configure ProjectB pipeline settings to allow it to connect to ProjectA in order to download the artifact that was produced.

pipesettings

Metadata for a pipeline resource, are available as predefined variables that we can reference, as you can see from our PipelineB.yml in the following code snippet:

## code/PipelineB.yml#L29-L30
script: |
  Write-output "This pipeline has been triggered by: $(resources.pipeline.PipelineA.pipelineName)"
Enter fullscreen mode Exit fullscreen mode

Predefined pipeline resource variables:

resources.pipeline.<Alias>.projectID
resources.pipeline.<Alias>.pipelineName
resources.pipeline.<Alias>.pipelineID
resources.pipeline.<Alias>.runName
resources.pipeline.<Alias>.runID
resources.pipeline.<Alias>.runURI
resources.pipeline.<Alias>.sourceBranch
resources.pipeline.<Alias>.sourceCommit
resources.pipeline.<Alias>.sourceProvider
resources.pipeline.<Alias>.requestedFor
resources.pipeline.<Alias>.requestedForID
Enter fullscreen mode Exit fullscreen mode

Now when we trigger and run PipelineA in ProjectA, it will automatically create our ArtifactA and also after completion PipelineB in ProjectB will be automatically triggered and also download and consume ArtifactA that was created in ProjectA.

results

Also note that triggers for resources are created based on the default branch configuration of our YAML, which is master. However, if we want to configure resource triggers from a different branch, we will need to change the default branch for the pipeline. For more information have a look at Default branch for triggers.

I hope you have enjoyed this post and have learned something new. You can also find the code samples used in this blog post on my Github page. ❤️

Author

Like, share, follow me on: 🐙 GitHub | 🐧 Twitter | 👾 LinkedIn

pwd9000 image

Discussion (0)