Joseph Pollack commited on
Commit
0b02a9a
·
unverified ·
1 Parent(s): bf1ec44

revery cursor fallback strategies on zerogpu decorator

Browse files
Files changed (2) hide show
  1. README.md +1 -1
  2. app.py +2 -13
README.md CHANGED
@@ -64,5 +64,5 @@ python download_bundles.py --inspect-only --infer # inspect existing downloads
64
 
65
  ## Deployment
66
 
67
- - **Hugging Face Spaces:** Set hardware to **Zero GPU** (or standard GPU). The app uses `@spaces.GPU` when available. If Zero GPU workers fail with *"No CUDA GPUs are available"* (e.g. no free GPU in the pool), set the **CPU_FALLBACK** secret or environment variable to `1` so inference runs on CPU in-process instead of in GPU workers.
68
  - **Local:** Run `python app.py`; requires a GPU with ~10 GB VRAM for all 6 models.
 
64
 
65
  ## Deployment
66
 
67
+ - **Hugging Face Spaces:** Set hardware to **Zero GPU** (or standard GPU). Decorate GPU-dependent functions with `@spaces.GPU` per [Spaces ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu).
68
  - **Local:** Run `python app.py`; requires a GPU with ~10 GB VRAM for all 6 models.
app.py CHANGED
@@ -18,18 +18,7 @@ from ui_strings import get_strings
18
  STARTUP_EXAMPLE_PROMPT = "dites moi en plus sur les jardins japonnais a paris :"
19
  STARTUP_EXAMPLE_SYSTEM = ""
20
 
21
- # Optional: use @spaces.GPU for ZeroGPU deployment.
22
- # Set CPU_FALLBACK=1 (or any non-empty value) to skip ZeroGPU and run inference on CPU
23
- # when no GPU is available (avoids "RuntimeError: No CUDA GPUs are available" in workers).
24
- import os
25
-
26
- try:
27
- import spaces
28
-
29
- _use_cpu_fallback = os.environ.get("CPU_FALLBACK", "").strip().lower() in ("1", "true", "yes")
30
- GPU_DECORATOR = (lambda f: f) if _use_cpu_fallback else spaces.GPU
31
- except ImportError:
32
- GPU_DECORATOR = lambda f: f # no-op when not on Spaces
33
 
34
  def build_params_by_model(
35
  temp_baguettotron: float,
@@ -64,7 +53,7 @@ def build_params_by_model(
64
  return params_by_model
65
 
66
 
67
- @GPU_DECORATOR
68
  def generate_all(
69
  prompt: str,
70
  system_prompt: str,
 
18
  STARTUP_EXAMPLE_PROMPT = "dites moi en plus sur les jardins japonnais a paris :"
19
  STARTUP_EXAMPLE_SYSTEM = ""
20
 
21
+ import spaces
 
 
 
 
 
 
 
 
 
 
 
22
 
23
  def build_params_by_model(
24
  temp_baguettotron: float,
 
53
  return params_by_model
54
 
55
 
56
+ @spaces.GPU
57
  def generate_all(
58
  prompt: str,
59
  system_prompt: str,