Allow templating the UV_PROJECT_ENVIRONMENT path#14937
Conversation
|
e.g., |
|
Is there a binary I can test? |
|
There are debug binaries attached to the bottom of the CI summary at https://github.com/astral-sh/uv/actions/runs/16572262146?pr=14937 |
|
I think this is pretty neat. I'm supportive of it, if you want to add docs etc. |
|
Tested on 3 different HPC setups, and works great, thanks! The only thing that I feel is missing (but probably it's just because I am not aware of how to do it) is a simple command to programmatically resolve the path to the current environment, som
8000
ething like
|
|
Environment variables are already complex and have interactions with e.g. the user's shell and their workspace/deploy path layout, could we skip over the templating the whole path and go directly to centralized env management the way e.g. poetry does it? This avoids exposing this complexity to users. Specifically, the complexity of templating the whole path (not just the leaf directory). |
The goal is to only expose this feature to motivated power users to start, so I don't mind some complexity here while we figure out how to make centralized environments a first-class feature. Do you have an example of what you're concerned about? Would it be resolved by adding a cache bucket and a |
|
We're introducing a new templating syntax that is only used in this one env var, we're adding a new concept just for this one place. If we want to experiment with centralized storage, why not start with a setting for a centralized root and e.g. a path hash and expand from that? If users want to use this for global configuration, we should primarily expose this in |
|
I guess one question is: if we do ship a centralized environment setting, would we continue to support this? If so, why? If not, maybe we just rip it on a setting. (I suspect it’s easy to implement, even easier than this.) |
I think this is a fair concern, though I'm not sure it's a big deal to support templating here. I'm not sure where else templating would be needed in uv so far, so it makes sense it doesn't exist as a concept today.
Yeah, I think so. The discussion at #14628 made it clear people want control over the path. I initially considered that out of scope for a first pass at the feature, but we probably would want it eventually.
I'm explicitly trying not to add new settings here. I'm not even sure what the top-level setting would look like, I guess some hybrid boolean path option? I think being able to control the root path is critical. We could certainly start with a boolean though.
A setting implies first-class support, which is what I'm trying to avoid getting into here. e.g., how does an editor discover the environment? how do I inspect or clean up environments? etc. The motivation for using a low-level environment variable is to avoid answering those questions while delivering some functionality now. |
(I guess the minimal thing would be an env var and
(I don't find the templating part to be that compelling based on that conversation, but I'm not strictly against it.)
I totally understand the motivation. Is there some risk, though, that by shipping this without solving these problems, we actually make it harder for us to solve them in the future? For example, if the paths are ~arbitrary (user-specified), how could an editor ever discover them? (How does that discovery work in Poetry, etc.?) By shipping this, we impose some constraints on ourselves that we either need to roll back or workaround later. |
Yeah, but I think we'd also want the root path to have a sensible default? Otherwise, it's kind of annoying since we don't do templating you'll have to hard-code an absolute path that's brittle?
I'm not sure I mind if users that opt in to arbitrarily complicated paths also need to configure their editor to find them. I think discovery works in Poetry via editor specific integrations. e.g.,
which comes down to parsing |
|
I guess I'll defer this and prototype an alternative approach later, my goal here was to pitch a quick idea that I thought might unblock people, but it's not clear we have consensus this is worth it. |
FWIW, I suggested
I've never felt like environment variables are any more or less complex than settings. Just a As a power user who is already hacking together an environment based solution with Mise, either
Yes, please! :)
I can't speak to the implementation, but I like the proposal and would love to have it sooner You've marked it as a preview. Does that give you any more confidence in shipping something like FWIW, mise has an "experimental" flag and puts some functionality behind that so that features that |
|
i cant find the issue but there were discussions about templating auth secrets for private indexes at some point. Whether or not you want to do that idk, but that's at least another theoretical usecase. What I like about this is that it's completely orthogonal to a more dedicated feature/setting, and I could imagine a future setting being implemented essentially in terms of this feature. e.g. you might eventually have some # settings.toml
managed_venv = true
managed_venv_root = "{venv_cache}" # default setting, saythat is essentially like setting a fallback And makes straightforward sense that in the presence of a user defined Not that this all couldnt be done similarly with a static setting without templating at all, but it seems like a lot more thought has to go into ensuring you're making everyone happy with a setting with fixed semantics. versus this, where everyone can, for sure, make the setting do what they want. As far as making this consumable, i feel like |
Is this dead then? Should it be closed? |
|
While this doesn't solve editor/tool environment discovery, my understanding is that discovery is already broken by setting Centralized envs are a highly desired feature by anyone using a cluster of machines where project source is stored on a network drive, which is all of HPC right now, and I think using an environment variable is totally fine for this use case. In the future, if there's a central configuration variable for a centralized environment path, it seems fine to me to override that with whatever is set in Are there any other potential downsides to this templating approach? To me, this appears to be pretty elegant and solves the issue without breaking anything that's not already broken by |
|
Any updates on this direction? I still think this is a great non-disruptive way of solving this somewhat urgent issue. |
I'm looking at supporting centralized environments, and this feels like a nice incremental step towards that. Basically we can re-use
UV_PROJECT_ENVIRONMENTto empower centralized storage without adding any more settings. Later, we'd add a toggle that enables centralized storage in the cache using our "canonical" recommended naming scheme. At that point, we'd probably explore things like empowering editor discovery via.venvpointers and such.The "canonical" usage would be something like:
Unfortunately, templating the Python version (which is a common request) is a little annoying because we determine the path to the environment before discovering a Python interpreter, but I got it working.
The only thing that's missing here is documentation, which I can add if there's consensus this is a good path forward.