How to treat packages without an OSI-approved license?

Is the question here about binary artifacts, or Julia source code?

For binary artifacts, yes we have the MKL artifacts (MKL_jll) and the CUDA artifacts (CUDA_jll), neither of which are open-source.

For Julia source code, we have historically always required an OSI license. My personal opinion is that we should preserve that requirement for Julia source code (although I can't speak for all General registry maintainers).

So, why should Julia source code be treated differently? For example if I compile a julia code with juliac and make an artifact, would different rules apply?

For example if I compile a julia code with juliac and make an artifact, would different rules apply?

In that case, if the original Julia source code wasn't available under an OSI-approved license, then in my opinion we shouldn't allow the corresponding binary artifacts in the General registry.

In my opinion, there's a couple different reasons for treating Julia source code differently.

1. Pragmatic: For MKL or CUDA, there's no realistic chance that we'd be able to convince Intel or NVIDIA to open-source them, even if we told them we'd remove MKL or CUDA from the General registry. In contrast, for Julia packages, we can actually ask package authors to open-source their packages, because package authors are usually active participants in the Julia community.
2. Philosophical: What is the purpose of the General registry? In my opinion, I think that the purpose of the General registry should be to distribute open-source Julia packages and to foster the growth of the open-source Julia ecosystem. Requiring Julia source code to be open-source helps further this mission.

but I was wondering if there's also any short term solution to get BonitoBook easy to install and depend on.

Can you give some more details on the specific pain points that you're running into? I do want it to be easy for people to use unregistered packages.

For simple usage, installing by URL is pretty easy right?

] add https://github.com/SimonDanisch/BonitoBook.jl

And for more complicated use cases, you can use the [sources] section of Project.toml, which IIUC is designed exactly for the use case of unregistered packages - and doesn't require distributing Manifest.toml.

Other solutions include:

1. Distributing Manifest.toml files.
2. Custom registry (which may be suitable for complicated commercial use cases).

Are there use cases that you're running into that aren't covered by the above?

Or are there pain points you're hitting with the above solutions? Perhaps there are improvements we can make to the above features?

Custom registry for one package creates worse UX than URL installation.

Could you say a little more about this? To me, naively it seems that the custom registry workflow is pretty simple (just two lines):

] registry add https://custom-registry-url

] add MyPackageName

To me, that seems pretty similar to the URL installation workflow (] add https://package-url).

call me naive, but it's literally open source and free to distribute ;) just not with unlimited usage for someone earning lots of money with the software ... Anyways the whole pipeline of registering a new version is super polished with General, catching mistakes, running tests automatically creating tags with release notes, etc... I guess the main question is, should this service apply to a software which is free to 95% of the Julia community instead of 100%. It would actually be a pleasure to put in some money to finance this amazing infrastructure, if i ever earn substantial money from this, so maybe this could be a way to bolster general? might be a slippery slope though, and hard to do right legally 🤷

The source section is nice, but it's also awkward to work with, im not even sure how it works with version conflicts and resolving? I would expect it not to take part in pkgs compat bound resolution? Also, I got a complaint that installing from source doesn't work for their company firewall ( not actually sure what's the problem there and why it works once it's registered).

1. Pragmatic: For MKL or CUDA, there's no realistic chance that we'd be able to convince Intel or NVIDIA to open-source them, even if we told them we'd remove MKL or CUDA from the General registry. In contrast, for Julia packages, we can actually ask package authors to open-source their packages, because package authors are usually active participants in the Julia community.

I feel this is actually not pragmatic - penalizing everyone else who is not big is not a good reason. Also, it feels a bit arbitrary, in that how do we decide what classifies as big. How about Gurobi_jll? That is a commercial solver that is already in the General registry, and you couldn't convince them to open source Gurobi either.

2. Philosophical: What is the purpose of the General registry? In my opinion, I think that the purpose of the General registry should be to distribute open-source Julia packages and to foster the growth of the open-source Julia ecosystem. Requiring Julia source code to be open-source helps further this mission.

I would be more onboard with a philosophical mission, but in that case, it would be best to apply it uniformly and remove MKL, CUDA, Gurobi and any others. The users clearly want these, and not allowing these would perhaps even reduce the size of the community.

I believe the purpose of the General registry is to make software available to Julia users in an easy way. In fact one could argue the mission is better served by allowing broader distribution, which would make it possible for more companies and people to choose Julia, which would create a vibrant community of Julia devs.

[Forgive the delay in response - I'm out of office a few days this week, so I'll write up some more thoughts when I'm back.]

One thing worth considering is if we should do a separate registry like Debian’s non-free.

Just cross referencing @ericphanson idea of a tag system as a possible solution: https://discourse.julialang.org/t/should-general-have-a-guideline-or-rule-preventing-registration-of-vibe-coded-packages/133205/52?u=sdanisch

I think there is a point in distinguishing between Julia source code and artifacts. Julia is a very compositional and interactive language. You can’t @edit your way into a binary or @less and see some transitive dependency code that isn’t open source. Or @eval a monkey patch.

Sure, even OSI code comes with conditions, but it at least promises you can use, modify, and share the code. With a non-OSI license we may be able to distribute it, but you the user may not be able to use it or modify it.

It feels like a trap! This code is in your session, fully and dynamically composing with your program, but you don’t have your usual rights.

The other way it’s different is just that the actual point of General, its reason to exist, is to distribute Julia code. It just seems fine if there are different rules for artifacts because they are incidental. I know legally it doesn’t really matter though.

Also, I think you’re aware, the Julia source code of Gurobi_jll (that thin shell) is indeed MIT licensed.

—-

Since non-OSI Julia code is currently not permitted though, if we do want to go this path we have the option of making it opt-in. Eg using tags as @SimonDanisch wrote, we could have Pkg only use the OSI-license tagged subset by default and require setting a preference to get non-OSI code.

I guess we might also want a stricter tag for all code and all artifacts are also freely licensed.

Sure, even OSI code comes with conditions, but it at least promises you can use, modify, and share the code. With a non-OSI license we may be able to distribute it, but you the user may not be able to use it or modify it.

By this definition, @SimonDanisch 's code meets the requirements and should be included. The fact that certain users have to pay for it does not take away their other rights (if I have understood the license correctly).

The other way it’s different is just that the actual point of General, its reason to exist, is to distribute Julia code. It just seems fine if there are different rules for artifacts because they are incidental. I know legally it doesn’t really matter though.

While I understand the sentiment - this now means that I am better off either writing my code in a non-Julia langauge and compiling a binary for inclusion in General as an artifact, or perhaps even creating a binary with juliac.

Also, I think you’re aware, the Julia source code of Gurobi_jll (that thin shell) is indeed MIT licensed.

Yes. My point was about the artifact - and to highlight the point that we do distribute non open source software that is not as widely used as MKL or CUDA.

Since non-OSI Julia code is currently not permitted though, if we do want to go this path we have the option of making it opt-in. Eg using tags as @SimonDanisch wrote, we could have Pkg only use the OSI-license tagged subset by default and require setting a preference to get non-OSI code.

I guess we might also want a stricter tag for all code and all artifacts are also freely licensed.

I like these suggestions as they would bring uniformity.

I am personally not advocating a specific solution. My point is mainly about consistency and uniformity, and currently things are a bit all over the place.

Yeah, I agree it’s a somewhat arbitrary line. But if you do write your code in another language or with juliac and call into from an open source wrapper, there is at least a clear technical line in your Julia session where the non-OSI code lies (on the other side of the ccalls). I guess with sufficient technical work one could blur this line and make @edit and other introspections dig even into the binary artifact if it’s Julia code, somehow, and merge method tables and stuff like that. And that would kind of defeat the separation. But it seems like it currently is a real separation.

The other issue I have is then philosophy of maintainership. If General is about serving other people’s paid-for software it feels weird to volunteer to do that work for free. It feels like eventually we’d have to move to some model like a foundation that hires a couple people and tries to scrape funding from big corporate users etc.

I’m not saying I wouldn’t work on it anymore but I would definitely need to think hard about it at least. I’m not sure how other maintainers feel about it. Maybe it’s not that different from folks using a package I wrote for a corporate use case or something like that. It kind of feels different though. Especially if there’s any change in expectations (“we’re losing money bc tool X is down, go fix it!!”).

I think the line is clear for a developer - but not really clear for an end user who doesn't look deep down into the dependency stack. With a solution like tags, it would be very clear to the end user what they are opting into.

I agree with the philosophy of maintenance - and perhaps it is arguable that MKL and CUDA are used so widely that maybe we (the open source community) take on the ownership - but Gurobi for example is niche and we do work with it. In other communities, homebrew does distribute casks and debian does have non-free. Perhaps the infrastructure can be common, so long as companies maintain the software they put into General.

Ah, I meant the maintenance of the registry and registry tooling, eg the folks who triage PRs to General, and work on its workflows, and packages like AutoMerge, RegistryCI, RegistryTools, TagBot, as well as the service Registrator. (Basically all this stuff: https://ericphanson.com/blog/2024/julias-package-registration-tooling/ + https://github.com/JuliaRegistries/General/blob/master/CONTRIBUTING.md).

Our current system (from the post of view of AutoMerge/RegistryCI) is for open source volunteers to build tools that can be used for both, but only actually deploy and maintain the instantiation only for General which is currently open source Julia packages.

Briefly looking into how Debian does it, it seems volunteers don’t really mind building / maintaining / deploying infra for both, so maybe this is not a real obstacle, not sure.

Agreed companies should maintain their own packages.

I have more thoughts to write up, and I'll try to write them up next week, but for now I wanted to address this:

Sure, even OSI code comes with conditions, but it at least promises you can use, modify, and share the code. With a non-OSI license we may be able to distribute it, but you the user may not be able to use it or modify it.

By this definition, @SimonDanisch 's code meets the requirements and should be included. The fact that certain users have to pay for it does not take away their other rights (if I have understood the license correctly).

I just want to clarify: the General registry's policy is specifically that we require an OSI-approved license. (This has been the documented policy for at least 10 years).

The idea is that we don't want registry maintainers to be in the position of litigating specific licenses (or litigating the definition of the phrase "Open Source"). Instead, we let the OSI do that work for us. We use the OSI's definition of "Open Source", and we use the OSI's list of licenses. If the license isn't on the OSI's list, then for the purposes of the General registry, that license is not "Open Source".

By this definition, @SimonDanisch 's code meets the requirements and should be included. The fact that certain users have to pay for it does not take away their other rights (if I have understood the license correctly).

I just want to clarify: the General registry's policy is specifically that we require an OSI-approved license. (This has been the documented policy for at least 10 years).

This is a good point, which I had forgotten.

The idea is that we don't want registry maintainers to be in the position of litigating specific licenses (or litigating the definition of the phrase "Open Source"). Instead, we let the OSI do that work for us. We use the OSI's definition of "Open Source", and we use the OSI's list of licenses. If the license isn't on the OSI's list, then for the purposes of the General registry, that license is not "Open Source".

Completely agree that this simplifies our decision making in the case of new licenses.

I still feel we need a coherent policy for JLLs, if it will be separate from Julia source packages.