RFC @cncf/toc, any volunteers to handle due diligence? cc: @amye

I can help on the DD if it is not very urgent.

@xiang90 we created the Tech-DD document and its pretty comprehensive in aiding for your review. I will send a link to it tomorrow morning. we are trying to button up approvals from a couple of customers to include them in the reference section.

Harbor will present in the November 5th, 2019 CNCF TOC call. As part of this review, we would like Harbor to be considered for Graduation.

@caniszczyk @monadic @lizrice For graduation project, should the project be reviewed by the SIG first?

@xiang90 , i added a link to the Tech-DD document that we prepared. https://docs.google.com/document/d/15gX7EeeXQThEvVMGpL-0a1mOwGuByLtMfvXNJaKT0A0/edit?usp=sharing

Please note that we are scheduled for a TOC review on Nov 5th. Is that sufficient time for you to review this document and do the due diligence?

Harbor is applying for graduation based on the v1.2 of the Graduation Criteria. I am not sure if that matters on the SIG review...

thank you!

@xiang90 which SIG is most appropriate, storage?

@xiang90 @caniszczyk Harbour falls within SIG-Runtime charter. Also, I think that sig-storage should review the storage-related aspects, and sig-security the security-related ones. I can co-ordinate/lead this from SIG-Runtime.

The following paragraph in the SIG-runtime charter pertains:

Note regarding Container Registries/Repositories and the like: While image packaging and distribution (and hence container registries/repositories in general) fall within the scope of this Runtime SIG, many of their common features and use cases are better dealt with by other CNCF SIGS. For example:

1. image storage, caching, etc - Storage SIG
2. Image encryption, signing etc - Security SIG
3. use of image registries to store and distribute many other types of artifacts, and in particular the format of these artifacts, including helm charts, OPA policies, public security certificates, data sets, machine learning models, etc, etc - the SIG relevant to those artifact types, e.g. Apps SIG, Security SIG, etc.

@xiang90 @caniszczyk Harbour falls within SIG-Runtime charter. Also, I think that sig-storage should review the storage-related aspects, and sig-security the security-related ones. I can co-ordinate/lead this from SIG-Runtime.

@quinton-hoole how would you like to proceed in this review? We have the technical due diligence document that offers a great start to get to know Harbor. we are available to meet and discuss any concerns or even to kick things off. please let me know. thanks!

I've created http://bit.ly/harbor-graduation-dd to consolidate the SIG reviews.

@quinton-hoole can we also please assign owners for this review and establish a timeframe as well?
I understand that you are a chair in both storage and runtime SIGs, so essentially you may be representing both SIGs.

@pragashj, @ultrasaurus, @dshaw, who can do this review and due diligence from sig-security?
@mattfarina and @bryanl , who can do this review from the sig-app-delivery?

thanks!

@michmike Yes, I can follow up on this. Everyone is pretty occupied with KubeCon coming up in a week, so this might take a while.

@michmike @caniszczyk

The DD looks solid. I made a few minor comments that should be addressed soon. One suggestion I have for Harbor is to create a maintainer diversity and encouragement plan, since most of the active maintainers are from VMWare.

For the SIG review, I suggest to have a single meeting/review for all SIGs reviewers. Or it will be too much burden for both Harbor and the SIGs.

@quinton-hoole the current DD is here https://docs.google.com/document/d/15gX7EeeXQThEvVMGpL-0a1mOwGuByLtMfvXNJaKT0A0/edit?usp=sharing

thank you @xiang90. We appreciate the thorough review. I will make the updates to the doc as per your suggestion.
In terms of the sig-review, the Harbor team is good to participate in one or more reviews with the relevant SIGs. Who will/can schedule them?

@quinton-hoole can we schedule some time to move this review along? thanks in advance!

happy new year everyone. @quinton-hoole , can we set up some time to move along this process?

Update: we reviewed the DD in SIG-Runtime and it looks very solid. @michmike has also addressed all the comments and concerns in the DD document. We recommend graduation: cncf/tag-runtime#7

CNCF SIG Storage Due Diligence Report can be found here. tldr; Some concerns were raised, leaving it up to TOC to determine if they are blocking or not.

@michmike Can you resolve the merge conflict?

@caniszczyk We are ready to call for a vote. The DD is ready. We discussed the dependency issue raised by SIG Storage. The majority of TOC believes that having external dependencies on popular non-CNCF open-source projects (like Redis or pgsql) are fine.

Harbor is now in public comment period, the vote will be called on May 26th.

Could someone please let us know why this submission needs to be prioritised over others (especially NATS)? Why is this submission not challenged the same way as NATS ("the requirement for a graduated project not to be under the sole control of one organisation has been an issue")

https://harbor.devstats.cncf.io/d/4/company-statistics-by-repository-group?orgId=1&from=now-5y&to=now&var-period=q&var-metric=activity&var-repogroup_name=All&var-companies=All

Cc @lizrice @michelleN @quinton-hoole @ColinSullivan1 @derekcollison @lucperkins

The TOC can prioritize things to their liking and judgement, they have final authority here. An analogy here is the US Supreme Court, I may not like every ruling but they get to choose what cases they accept and their priority.

For maintainer diversity, I think it's important to look at a few things, the link you mentioned above is a good one along with the official maintainer list and how receptive the project has been in adding new maintainers over time: http://maintainers.cncf.io and also a projects direct maintainers governance: https://github.com/goharbor/community/blob/master/MAINTAINERS.md

Also if you look at our project health dashboards, you can see things a bit more clearly if a project is behaving in a healthy manner:
https://all.devstats.cncf.io/d/54/project-health-table?orgId=1&var-repogroup_name=NATS
https://all.devstats.cncf.io/d/54/project-health-table?orgId=1&var-repogroup_name=Harbor

@caniszczyk Thanks for your reply, I believe the particular concern @lizrice raised was about the "sole control of one organisation" not about health. Even in the health link's data, it indicated how solely one particular company controls this project. What is the acceptable sole control for a graduated project? Or is it just about names in a particular document? In another submission, I've noticed that the maintainers in the document were not "Technically" maintainers ( #379 (comment) )

Just wanted to quickly share some insights about the great job done by the Harbor team in the last year if that can contribute to TOC appreciation.

I am PM for a large Europe-based cloud provider. When we experimented with Harbor a year ago as a possible solution for our registry product, starting with great feedback from customers and prospects using Harbor onprem and in enterprise deployments (custom or as part of some entrerprise suite). The project was clearly mature on the functional basis, but it still had room for improvement in two areas: documentation and control by one company. Both of those concerns have been resolved by the project since.

We quickly entered the community (and the great bi-weekly community meeting and reactivity on the Slack channels and in 1-1 basis made it easy) and were quickly reassured by the fact that the team had its sights on a common vision with us and were in fact addressing some of our concerns.

After months of working with the team, issues, PRs, and discussions, we contributed to the community what is now integrated asthe official Harbor operator (currently in v0.5 as a subfolder in the project repo).

Our company now has a stake in the Harbor project, adding even more diversity than what was already happening in the last year, and we saw a big improvement around the documentation effort in the last 6 months.

I think Harbor is very stable and massively used in the field, and even though the registry is being often considered a commodity, Harbor has its place as a key component in our solution and we are very happy with the community and resulting product.

That's really great to hear. Thanks for the testimonial @mhurtel

@mhurtrel Thank you for your contribution to the project. The harbor project has made an important change by replacing the Clair with Aqua’s Trivy as the default image scanner. I believe Clair was the default scanner for 3+ years. Has this change been communicated to the harbor users via the community channels and requested feedback? Also, does the harbor team know if the Trivy project has any plans to join the CNCF similar to Clair?

Yes the change from Clair to Trivy was brought up over 10 times in community meetings, planning meetings for the 2.0 release and in blog posts. None of our users or contributors raised any concerns. In contrary, the change was celebrated. I can't speak to Trivy's goals around CNCF. You can ask Liz Rice in private.

Be aware that Clair was not deprecated. It is still a built in component that's deployed with Harbor. It is just that now Trivy is the default scanner.

@michmike Thank you for your reply. Has this ever been notified in the Harbor mailing lists asking for feedback? I don't remember seeing one, such notifications would be really helpful.

Also, I think if the website will explicitly mention details of all projects that are vertically integrated to offer the core features ( Security and vulnerability analysis: Clair, Trivy, etc., Content signing and validation: Notary, etc.) in Harbor will help to reduce user confusion.

Hey @michmike can you fix the the conflicts in the branch?

Harbor has enough votes to graduate:

+1 Binding: note: Quorum is 10 as Jeff Brewer has been away
7 TOC votes:
Sheng Liang: https://lists.cncf.io/g/cncf-toc/message/4773
Xiang Li: https://lists.cncf.io/g/cncf-toc/message/4775
Katie Gamanji: https://lists.cncf.io/g/cncf-toc/message/4784
Justin Cormack: https://lists.cncf.io/g/cncf-toc/message/4800
Liz Rice: https://lists.cncf.io/g/cncf-toc/message/4816
Alena Prokharchyk: https://lists.cncf.io/g/cncf-toc/message/4818
Michelle Noorali: https://lists.cncf.io/g/cncf-toc/message/4836
Saad Ali: https://lists.cncf.io/g/cncf-toc/message/4840

working on this... :)

@caniszczyk you can merge it now

Thanks @michmike and welcome to the graduated class!