Unable to sync images from hub.docker.com


email.kprasad@...
 

Hi All,

I'm trying to test image replication from hub.docker.com but nothing happens after clicking "REPLICATE" button. The following are settings:

Harbor Version v1.7.5-a8f6543a

Under Registries:
Under Replications:
  • Source images filter : "repository"  r/andyshinn/dnsmasq

In the "source images filter" field I tried various combinations of repo name, like just "dnsmasq" and "andyshinn/dnsmasq". I also tried replicating different repo like "_/python", but still facing the same issue, not able to replicate. Any idea what could be the issue ?

Regards,
Prasad



Steven Zou
 

Can you try the following pattern:

 

  Repository : library/redis*

  Tag: latest

 

 

And check what happen?

 

thanks&regards

-- 

Steven Zou(邹佳)

 

Engineer, MAP(Modern Application Platform), VMware R&D | Harbor Maintainer

 

Mail: szou@...

GitHub: github.com/steven-zou

Cell: +8618600021252

Addr: 9F Tower C, Raycom Info Tech Park, No. 2 Kexueyuan South Road Haidian District, Beijing 100738 China

 

 

 

From: <harbor-users@...> on behalf of "email.kprasad via Lists.Cncf.Io" <email.kprasad=gmail.com@...>
Reply-To: "harbor-users@..." <harbor-users@...>
Date: Tuesday, February 18, 2020 at 13:08
To: "harbor-users@..." <harbor-users@...>
Subject: [harbor-users] Unable to sync images from hub.docker.com

 

Hi All,

I'm trying to test image replication from hub.docker.com but nothing happens after clicking "REPLICATE" button. The following are settings:

Harbor Version v1.7.5-a8f6543a

Under Registries:

Under Replications:

  • Source images filter : "repository"  r/andyshinn/dnsmasq


In the "source images filter" field I tried various combinations of repo name, like just "dnsmasq" and "andyshinn/dnsmasq". I also tried replicating different repo like "_/python", but still facing the same issue, not able to replicate. Any idea what could be the issue ?

Regards,
Prasad

 


email.kprasad@...
 

Hi Steven,

I tired but still the same result. I also checked the logs and could find only this :
-----
Feb 18 06:46:00 host-name core[12699]: 2020-02-18T06:46:00Z [INFO] replication signal for policy 5 sent
Feb 18 06:46:00 host-name core[12699]: 2020/02/18 06:46:00 #033[1;44m[D] [server.go:2619] |  182.74.75.171|#033[42m 200 #033[0m|   4.135711ms|   match|#033[46m POST    #033[0m /api/replications   r:/api/replications#033[0m
Feb 18 06:46:00 host-name core[12699]: 2020-02-18T06:46:00Z [INFO] Handle notification with topic 'StartReplication': notification.StartReplicationNotification{PolicyID:5, Metadata:map[string]interface {}{"op_uuid":"539253553bd54e728ac9d79c39a214e3"}}
-----

Also I just noticed, the registry endpoint which I added for DockerHub, on clicking "Test Connection" it says failed to ping endpoint. Could this be the issue ? But I'm able to curl the endpoint from the harbor node successfully, so connectivity is not an issue.
----
root [ /harbor ]# curl -I https://hub.docker.com
HTTP/1.1 200 OK
Date: Tue, 18 Feb 2020 06:55:49 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 2110
ETag: W/"83e-soAKjyBJXD/TFDFDjBU9dyLtT5o"
Vary: Accept-Encoding
Server: nginx
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000
-----

Thanks,
Prasad



Steven Zou
 

Test connection failed means the harbor service cannot connect to the docker hub and then the replication will be definitely failed.  For replicating images under library, you do not need to provide credentials. You can clear the credential and do “Test Connection” again (uncheck the “Verify remote cert” too).

 

As far as I know, docker hub replication adapter is working well so far.

 

If the issue is still existing, you can raise an issue in our GitHub repo and provide related contexts/logs for debugging.

 

thanks&regards

-- 

Steven Zou(邹佳)

 

Engineer, MAP(Modern Application Platform), VMware R&D | Harbor Maintainer

 

Mail: szou@...

GitHub: github.com/steven-zou

Cell: +8618600021252

Addr: 9F Tower C, Raycom Info Tech Park, No. 2 Kexueyuan South Road Haidian District, Beijing 100738 China

 

 

 

From: <harbor-users@...> on behalf of "Prasad K via Lists.Cncf.Io" <email.kprasad=gmail.com@...>
Reply-To: "harbor-users@..." <harbor-users@...>
Date: Tuesday, February 18, 2020 at 14:57
To: "harbor-users@..." <harbor-users@...>
Subject: Re: [harbor-users] Unable to sync images from hub.docker.com

 

Hi Steven,

 

I tired but still the same result. I also checked the logs and could find only this :

-----

Feb 18 06:46:00 host-name core[12699]: 2020-02-18T06:46:00Z [INFO] replication signal for policy 5 sent

Feb 18 06:46:00 host-name core[12699]: 2020/02/18 06:46:00 #033[1;44m[D] [server.go:2619] |  182.74.75.171|#033[42m 200 #033[0m|   4.135711ms|   match|#033[46m POST    #033[0m /api/replications   r:/api/replications#033[0m

Feb 18 06:46:00 host-name core[12699]: 2020-02-18T06:46:00Z [INFO] Handle notification with topic 'StartReplication': notification.StartReplicationNotification{PolicyID:5, Metadata:map[string]interface {}{"op_uuid":"539253553bd54e728ac9d79c39a214e3"}}

-----

 

Also I just noticed, the registry endpoint which I added for DockerHub, on clicking "Test Connection" it says failed to ping endpoint. Could this be the issue ? But I'm able to curl the endpoint from the harbor node successfully, so connectivity is not an issue.

----

root [ /harbor ]# curl -I https://hub.docker.com

HTTP/1.1 200 OK

Date: Tue, 18 Feb 2020 06:55:49 GMT

Content-Type: text/html; charset=utf-8

Content-Length: 2110

ETag: W/"83e-soAKjyBJXD/TFDFDjBU9dyLtT5o"

Vary: Accept-Encoding

Server: nginx

X-Frame-Options: deny

X-Content-Type-Options: nosniff

X-XSS-Protection: 1; mode=block

Strict-Transport-Security: max-age=31536000

-----

 

Thanks,

Prasad

 

 


Prasad K <email.kprasad@...>
 

Yes, I'm replicating images under library. I cleared the credentials and did a test connection and got the same error message. This is what I found from the logs after clicking test connection:
----
Feb 18 07:11:09 host-name core[12699]: 2020-02-18T07:11:09Z [WARNING] Schemas [] are unsupported
Feb 18 07:11:09 host-name core[12699]: 2020-02-18T07:11:09Z [WARNING] empty realm, skip
Feb 18 07:11:10 host-name core[12699]: 2020-02-18T07:11:10Z [ERROR] [target.go:65]: failed to ping target: 404
----

The endpoint URL is "https://hub.docker.com". Is there something missing in my configuration ?

Thanks,
Prasad

On Tue, 18 Feb 2020 at 12:37, Steven Zou via Lists.Cncf.Io <szou=vmware.com@...> wrote:

Test connection failed means the harbor service cannot connect to the docker hub and then the replication will be definitely failed.  For replicating images under library, you do not need to provide credentials. You can clear the credential and do “Test Connection” again (uncheck the “Verify remote cert” too).

 

As far as I know, docker hub replication adapter is working well so far.

 

If the issue is still existing, you can raise an issue in our GitHub repo and provide related contexts/logs for debugging.

 

thanks&regards

-- 

Steven Zou(邹佳)

 

Engineer, MAP(Modern Application Platform), VMware R&D | Harbor Maintainer

 

Cell: +8618600021252

Addr: 9F Tower C, Raycom Info Tech Park, No. 2 Kexueyuan South Road Haidian District, Beijing 100738 China

 

 

 

From: <harbor-users@...> on behalf of "Prasad K via Lists.Cncf.Io" <email.kprasad=gmail.com@...>
Reply-To: "harbor-users@..." <harbor-users@...>
Date: Tuesday, February 18, 2020 at 14:57
To: "harbor-users@..." <harbor-users@...>
Subject: Re: [harbor-users] Unable to sync images from hub.docker.com

 

Hi Steven,

 

I tired but still the same result. I also checked the logs and could find only this :

-----

Feb 18 06:46:00 host-name core[12699]: 2020-02-18T06:46:00Z [INFO] replication signal for policy 5 sent

Feb 18 06:46:00 host-name core[12699]: 2020/02/18 06:46:00 #033[1;44m[D] [server.go:2619] |  182.74.75.171|#033[42m 200 #033[0m|   4.135711ms|   match|#033[46m POST    #033[0m /api/replications   r:/api/replications#033[0m

Feb 18 06:46:00 host-name core[12699]: 2020-02-18T06:46:00Z [INFO] Handle notification with topic 'StartReplication': notification.StartReplicationNotification{PolicyID:5, Metadata:map[string]interface {}{"op_uuid":"539253553bd54e728ac9d79c39a214e3"}}

-----

 

Also I just noticed, the registry endpoint which I added for DockerHub, on clicking "Test Connection" it says failed to ping endpoint. Could this be the issue ? But I'm able to curl the endpoint from the harbor node successfully, so connectivity is not an issue.

----

root [ /harbor ]# curl -I https://hub.docker.com

HTTP/1.1 200 OK

Date: Tue, 18 Feb 2020 06:55:49 GMT

Content-Type: text/html; charset=utf-8

Content-Length: 2110

ETag: W/"83e-soAKjyBJXD/TFDFDjBU9dyLtT5o"

Vary: Accept-Encoding

Server: nginx

X-Frame-Options: deny

X-Content-Type-Options: nosniff

X-XSS-Protection: 1; mode=block

Strict-Transport-Security: max-age=31536000

-----

 

Thanks,

Prasad