Re: Assistance testing Harbor for OCI conformance

Yan Wang

For the live testing, I have opened a PR at We’d like to enable the live testing as Harbor repo so that we can know which PR introduces new failures and fix it on time.


@Daniel Jiang @Alex Xu  your idea?





From: <harbor-dev@...> on behalf of "Josh Dolitsky via" <>
Reply-To: "harbor-dev@..." <harbor-dev@...>
Date: Wednesday, 29 April 2020 at 11:02 PM
To: "harbor-dev@..." <harbor-dev@...>
Subject: Re: [harbor-dev] Assistance testing Harbor for OCI conformance


Yan, thanks for helping with this. Do you know if it is possible to start a fresh new Harbor instance in GitHub actions? GitHub actions allows for a "services" section which you can launch containers for the test run. This is what we really need help from Harbor team on. Here is a example action using a redis:

Since Harbor is an opensource registry, it would be preferred to do it this way vs. testing a live version (i.e. This is definitely a great start regardless.

Thank you,





On Wed, Apr 29, 2020 at 4:41 AM Yan Wang <wangyan@...> wrote:

Hi Josh,


Thanks for your detailed info.


I have filed an PR to add Harbor conformance test, could you please help to review?

BTW, can you help to add two more secrets(Harbor_USERNAME and Harbor_PASSWORD) to store the testing account auth?





From: <harbor-dev@...> on behalf of "Daniel Jiang via" <>
Reply-To: "harbor-dev@..." <harbor-dev@...>
Date: Tuesday, 7 April 2020 at 2:05 AM
To: "harbor-dev@..." <harbor-dev@...>
Subject: Re: [harbor-dev] Assistance testing Harbor for OCI conformance




Thanks for the heads up.  The conformance test has been on our radar and we did some manual dry run locally.


Adding Harbor to the report is on the todo list, and we’ll have that incorporated within this or next week.


Best Regards


Daniel Jiang  | 姜坦

Engineer, VMware R&D, Beijing

+86 10-59934536



On 2020/4/6, 10:21 AM, "harbor-dev@... on behalf of Josh Dolitsky via" <harbor-dev@... on behalf of> wrote:


Hello Harbor team,

We're seeking help setting up test automation using GitHub actions to test Harbor for OCI conformance. See full issue here.

For background: the OCI Distribution Specification is an open spec based on Docker's Registry API, and is the current workspace for defining standards concerning container registries such as Harbor. There is a recent effort to enable registry providers to validate if a registry properly conforms to the specification. 


We need your expertise in setting up Harbor for tests purposes within the context of GitHub actions, as well as configuration setup for running the OCI conformance test suite.

You can view a live view of preliminary test results at Click one of the "test report" links to open up a detailed conformance report.

If you're interested in the larger discussion, I encourage you to join the the OCI mailing list.

Feel free to email me or ping me on Slack if you have any questions etc.


- Josh Dolitsky (@jdolitsky)

Join to automatically receive all group messages.