happyz
/
Fooocus
mirror of https://github.com/lllyasviel/Fooocus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into main

This commit is contained in:
Tomáš Dinh 2024-05-27 11:11:28 +02:00 committed by GitHub
parent 449149d092 dfff9b7dcf
commit c4ab21993a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
85 changed files with 3895 additions and 873 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
.dockerignore Normal file
View File

@ -0,0 +1,54 @@
__pycache__
*.ckpt
*.safetensors
*.pth
*.pt
*.bin
*.patch
*.backup
*.corrupted
*.partial
*.onnx
sorted_styles.json
/input
/cache
/language/default.json
/test_imgs
config.txt
config_modification_tutorial.txt
user_path_config.txt
user_path_config-deprecated.txt
/modules/*.png
/repositories
/fooocus_env
/venv
/tmp
/ui-config.json
/outputs
/config.json
/log
/webui.settings.bat
/embeddings
/styles.csv
/params.txt
/styles.csv.bak
/webui-user.bat
/webui-user.sh
/interrogate
/user.css
/.idea
/notification.ogg
/notification.mp3
/SwinIR
/textual_inversion
.vscode
/extensions
/test/stdout.txt
/test/stderr.txt
/cache.json*
/config_states/
/node_modules
/package-lock.json
/.coverage*
/auth.json
.DS_Store

3
.gitattributes vendored Normal file
View File

@ -0,0 +1,3 @@
# Ensure that shell scripts always use lf line endings, e.g. entrypoint.sh for docker
* text=auto
*.sh text eol=lf

2
.github/CODEOWNERS vendored
View File

@ -1 +1 @@
* @lllyasviel
* @mashb1t

18
.github/ISSUE_TEMPLATE/bug_report.md vendored
View File

@ -1,18 +0,0 @@
---
name: Bug report
about: Describe a problem
title: ''
labels: ''
assignees: ''
---
**Read Troubleshoot**
[x] I admit that I have read the [Troubleshoot](https://github.com/lllyasviel/Fooocus/blob/main/troubleshoot.md) before making this issue.
**Describe the problem**
A clear and concise description of what the bug is.
**Full Console Log**
Paste **full** console log here. You will make our job easier if you give a **full** log.

107
.github/ISSUE_TEMPLATE/bug_report.yml vendored Normal file
View File

@ -0,0 +1,107 @@
name: Bug Report
description: You think something is broken in Fooocus
title: "[Bug]: "
labels: ["bug", "triage"]
body:
- type: markdown
attributes:
value: |
> The title of the bug report should be short and descriptive.
> Use relevant keywords for searchability.
> Do not leave it blank, but also do not put an entire error log in it.
- type: checkboxes
attributes:
label: Checklist
description: |
Please perform basic debugging to see if your configuration is the cause of the issue.
Basic debug procedure
　2. Update Fooocus - sometimes things just need to be updated
　3. Backup and remove your config.txt - check if the issue is caused by bad configuration
　5. Try a fresh installation of Fooocus in a different directory - see if a clean installation solves the issue
Before making a issue report please, check that the issue hasn't been reported recently.
options:
- label: The issue has not been resolved by following the [troubleshooting guide](https://github.com/lllyasviel/Fooocus/blob/main/troubleshoot.md)
- label: The issue exists on a clean installation of Fooocus
- label: The issue exists in the current version of Fooocus
- label: The issue has not been reported before recently
- label: The issue has been reported before but has not been fixed yet
- type: markdown
attributes:
value: |
> Please fill this form with as much information as possible. Don't forget to add information about "What browsers" and provide screenshots if possible
- type: textarea
id: what-did
attributes:
label: What happened?
description: Tell us what happened in a very clear and simple way
placeholder: |
image generation is not working as intended.
validations:
required: true
- type: textarea
id: steps
attributes:
label: Steps to reproduce the problem
description: Please provide us with precise step by step instructions on how to reproduce the bug
placeholder: |
1. Go to ...
2. Press ...
3. ...
validations:
required: true
- type: textarea
id: what-should
attributes:
label: What should have happened?
description: Tell us what you think the normal behavior should be
placeholder: |
Fooocus should ...
validations:
required: true
- type: dropdown
id: browsers
attributes:
label: What browsers do you use to access Fooocus?
multiple: true
options:
- Mozilla Firefox
- Google Chrome
- Brave
- Apple Safari
- Microsoft Edge
- Android
- iOS
- Other
- type: dropdown
id: hosting
attributes:
label: Where are you running Fooocus?
multiple: false
options:
- Locally
- Locally with virtualization (e.g. Docker)
- Cloud (Google Colab)
- Cloud (other)
- type: input
id: operating-system
attributes:
label: What operating system are you using?
placeholder: |
Windows 10
- type: textarea
id: logs
attributes:
label: Console logs
description: Please provide **full** cmd/terminal logs from the moment you started UI to the end of it, after the bug occured. If it's very long, provide a link to pastebin or similar service.
render: Shell
validations:
required: true
- type: textarea
id: misc
attributes:
label: Additional information
description: |
Please provide us with any relevant additional info or context.
Examples:
　I have updated my GPU driver recently.

5
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@ -0,0 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: Ask a question
url: https://github.com/lllyasviel/Fooocus/discussions/new?category=q-a
about: Ask the community for help

14
.github/ISSUE_TEMPLATE/feature_request.md vendored
View File

@ -1,14 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the idea you'd like**
A clear and concise description of what you want to happen.

40
.github/ISSUE_TEMPLATE/feature_request.yml vendored Normal file
View File

@ -0,0 +1,40 @@
name: Feature request
description: Suggest an idea for this project
title: "[Feature Request]: "
labels: ["enhancement", "triage"]
body:
- type: checkboxes
attributes:
label: Is there an existing issue for this?
description: Please search to see if an issue already exists for the feature you want, and that it's not implemented in a recent build/commit.
options:
- label: I have searched the existing issues and checked the recent builds/commits
required: true
- type: markdown
attributes:
value: |
*Please fill this form with as much information as possible, provide screenshots and/or illustrations of the feature if possible*
- type: textarea
id: feature
attributes:
label: What would your feature do?
description: Tell us about your feature in a very clear and simple way, and what problem it would solve
validations:
required: true
- type: textarea
id: workflow
attributes:
label: Proposed workflow
description: Please provide us with step by step information on how you'd like the feature to be accessed and used
value: |
1. Go to ....
2. Press ....
3. ...
validations:
required: true
- type: textarea
id: misc
attributes:
label: Additional information
description: Add any other context or screenshots about the feature request here.

6
.github/dependabot.yml vendored Normal file
View File

@ -0,0 +1,6 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"

44
.github/workflows/build_container.yml vendored Normal file
View File

@ -0,0 +1,44 @@
name: Create and publish a container image
on:
push:
tags:
- 'v*'
jobs:
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/${{ github.repository_owner }}/${{ github.event.repository.name }}
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

1
.gitignore vendored
View File

@ -51,3 +51,4 @@ user_path_config-deprecated.txt
/package-lock.json
/.coverage*
/auth.json
.DS_Store

29
Dockerfile Normal file
View File

@ -0,0 +1,29 @@
FROM nvidia/cuda:12.4.1-base-ubuntu22.04
ENV DEBIAN_FRONTEND noninteractive
ENV CMDARGS --listen
RUN apt-get update -y && \
apt-get install -y curl libgl1 libglib2.0-0 python3-pip python-is-python3 git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
COPY requirements_docker.txt requirements_versions.txt /tmp/
RUN pip install --no-cache-dir -r /tmp/requirements_docker.txt -r /tmp/requirements_versions.txt && \
rm -f /tmp/requirements_docker.txt /tmp/requirements_versions.txt
RUN pip install --no-cache-dir xformers==0.0.23 --no-dependencies
RUN curl -fsL -o /usr/local/lib/python3.10/dist-packages/gradio/frpc_linux_amd64_v0.2 https://cdn-media.huggingface.co/frpc-gradio-0.2/frpc_linux_amd64 && \
chmod +x /usr/local/lib/python3.10/dist-packages/gradio/frpc_linux_amd64_v0.2
RUN adduser --disabled-password --gecos '' user && \
mkdir -p /content/app /content/data
COPY entrypoint.sh /content/
RUN chown -R user:user /content
WORKDIR /content
USER user
COPY . /content/app
RUN mv /content/app/models /content/app/models.org
CMD [ "sh", "-c", "/content/entrypoint.sh ${CMDARGS}" ]

14
args_manager.py
View File

@ -1,8 +1,13 @@
import ldm_patched.modules.args_parser as args_parser
import os
from tempfile import gettempdir
args_parser.parser.add_argument("--share", action='store_true', help="Set whether to share on Gradio.")
args_parser.parser.add_argument("--preset", type=str, default=None, help="Apply specified UI preset.")
args_parser.parser.add_argument("--disable-preset-selection", action='store_true',
help="Disables preset selection in Gradio.")
args_parser.parser.add_argument("--language", type=str, default='default',
help="Translate UI using json files in [language] folder. "
@ -18,11 +23,17 @@ args_parser.parser.add_argument("--disable-image-log", action='store_true',
help="Prevent writing images and logs to hard drive.")
args_parser.parser.add_argument("--disable-analytics", action='store_true',
help="Disables analytics for Gradio", default=False)
help="Disables analytics for Gradio.")
args_parser.parser.add_argument("--disable-metadata", action='store_true',
help="Disables saving metadata to images.")
args_parser.parser.add_argument("--disable-preset-download", action='store_true',
help="Disables downloading models for presets", default=False)
args_parser.parser.add_argument("--enable-describe-uov-image", action='store_true',
help="Disables automatic description of uov images when prompt is empty", default=False)
args_parser.parser.add_argument("--always-download-new-model", action='store_true',
help="Always download newer models ", default=False)
@ -40,6 +51,7 @@ args_parser.args.always_offload_from_vram = not args_parser.args.disable_offload
if args_parser.args.disable_analytics:
import os
os.environ["GRADIO_ANALYTICS_ENABLED"] = "False"
if args_parser.args.disable_in_browser:
args_parser.args.in_browser = False

194
css/style.css
View File

@ -1,5 +1,146 @@
/* based on https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/v1.6.0/style.css */
.loader-container {
display: flex; /* Use flex to align items horizontally */
align-items: center; /* Center items vertically within the container */
white-space: nowrap; /* Prevent line breaks within the container */
}
.loader {
border: 8px solid #f3f3f3; /* Light grey */
border-top: 8px solid #3498db; /* Blue */
border-radius: 50%;
width: 30px;
height: 30px;
animation: spin 2s linear infinite;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
/* Style the progress bar */
progress {
appearance: none; /* Remove default styling */
height: 20px; /* Set the height of the progress bar */
border-radius: 5px; /* Round the corners of the progress bar */
background-color: #f3f3f3; /* Light grey background */
width: 100%;
vertical-align: middle !important;
}
/* Style the progress bar container */
.progress-container {
margin-left: 20px;
margin-right: 20px;
flex-grow: 1; /* Allow the progress container to take up remaining space */
}
/* Set the color of the progress bar fill */
progress::-webkit-progress-value {
background-color: #3498db; /* Blue color for the fill */
}
progress::-moz-progress-bar {
background-color: #3498db; /* Blue color for the fill in Firefox */
}
/* Style the text on the progress bar */
progress::after {
content: attr(value '%'); /* Display the progress value followed by '%' */
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
color: white; /* Set text color */
font-size: 14px; /* Set font size */
}
/* Style other texts */
.loader-container > span {
margin-left: 5px; /* Add spacing between the progress bar and the text */
}
.progress-bar > .generating {
display: none !important;
}
.progress-bar{
height: 30px !important;
}
.progress-bar span {
text-align: right;
width: 215px;
}
.type_row{
height: 80px !important;
}
.type_row_half{
height: 32px !important;
}
.scroll-hide{
resize: none !important;
}
.refresh_button{
border: none !important;
background: none !important;
font-size: none !important;
box-shadow: none !important;
}
.advanced_check_row{
width: 250px !important;
}
.min_check{
min-width: min(1px, 100%) !important;
}
.resizable_area {
resize: vertical;
overflow: auto !important;
}
.performance_selection label {
width: 140px !important;
}
.aspect_ratios label {
flex: calc(50% - 5px) !important;
}
.aspect_ratios label span {
white-space: nowrap !important;
}
.aspect_ratios label input {
margin-left: -5px !important;
}
.lora_enable label {
height: 100%;
}
.lora_enable label input {
margin: auto;
}
.lora_enable label span {
display: none;
}
@-moz-document url-prefix() {
.lora_weight input[type=number] {
width: 80px;
}
}
#context-menu{
z-index:9999;
position:absolute;
@ -218,3 +359,56 @@
#stylePreviewOverlay.lower-half {
transform: translate(-140px, -140px);
}
/* scrollable box for style selections */
.contain .tabs {
height: 100%;
}
.contain .tabs .tabitem.style_selections_tab {
height: 100%;
}
.contain .tabs .tabitem.style_selections_tab > div:first-child {
height: 100%;
}
.contain .tabs .tabitem.style_selections_tab .style_selections {
min-height: 200px;
height: 100%;
}
.contain .tabs .tabitem.style_selections_tab .style_selections .wrap[data-testid="checkbox-group"] {
position: absolute; /* remove this to disable scrolling within the checkbox-group */
overflow: auto;
padding-right: 2px;
max-height: 100%;
}
.contain .tabs .tabitem.style_selections_tab .style_selections .wrap[data-testid="checkbox-group"] label {
/* max-width: calc(35% - 15px) !important; */ /* add this to enable 3 columns layout */
flex: calc(50% - 5px) !important;
}
.contain .tabs .tabitem.style_selections_tab .style_selections .wrap[data-testid="checkbox-group"] label span {
/* white-space:nowrap; */ /* add this to disable text wrapping (better choice for 3 columns layout) */
overflow: hidden;
text-overflow: ellipsis;
}
/* styles preview tooltip */
.preview-tooltip {
background-color: #fff8;
font-family: monospace;
text-align: center;
border-radius: 5px 5px 0px 0px;
display: none; /* remove this to enable tooltip in preview image */
}
#inpaint_canvas .canvas-tooltip-info {
top: 2px;
}
#inpaint_brush_color input[type=color]{
background: none;
}

11
development.md Normal file
View File

@ -0,0 +1,11 @@
## Running unit tests
Native python:
```
python -m unittest tests/
```
Embedded python (Windows zip file installation method):
```
..\python_embeded\python.exe -m unittest
```

36
docker-compose.yml Normal file
View File

@ -0,0 +1,36 @@
volumes:
fooocus-data:
services:
app:
build: .
image: ghcr.io/lllyasviel/fooocus
ports:
- "7865:7865"
environment:
- CMDARGS=--listen # Arguments for launch.py.
- DATADIR=/content/data # Directory which stores models, outputs dir
- config_path=/content/data/config.txt
- config_example_path=/content/data/config_modification_tutorial.txt
- path_checkpoints=/content/data/models/checkpoints/
- path_loras=/content/data/models/loras/
- path_embeddings=/content/data/models/embeddings/
- path_vae_approx=/content/data/models/vae_approx/
- path_upscale_models=/content/data/models/upscale_models/
- path_inpaint=/content/data/models/inpaint/
- path_controlnet=/content/data/models/controlnet/
- path_clip_vision=/content/data/models/clip_vision/
- path_fooocus_expansion=/content/data/models/prompt_expansion/fooocus_expansion/
- path_outputs=/content/app/outputs/ # Warning: If it is not located under '/content/app', you can't see history log!
volumes:
- fooocus-data:/content/data
#- ./models:/import/models # Once you import files, you don't need to mount again.
#- ./outputs:/import/outputs # Once you import files, you don't need to mount again.
tty: true
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [compute, utility]

131
docker.md Normal file
View File

@ -0,0 +1,131 @@
# Fooocus on Docker
The docker image is based on NVIDIA CUDA 12.4 and PyTorch 2.1, see [Dockerfile](Dockerfile) and [requirements_docker.txt](requirements_docker.txt) for details.
## Requirements
- A computer with specs good enough to run Fooocus, and proprietary Nvidia drivers
- Docker, Docker Compose, or Podman
## Quick start
**More information in the [notes](#notes).**
### Running with Docker Compose
1. Clone this repository
2. Run the docker container with `docker compose up`.
### Running with Docker
```sh
docker run -p 7865:7865 -v fooocus-data:/content/data -it \
--gpus all \
-e CMDARGS=--listen \
-e DATADIR=/content/data \
-e config_path=/content/data/config.txt \
-e config_example_path=/content/data/config_modification_tutorial.txt \
-e path_checkpoints=/content/data/models/checkpoints/ \
-e path_loras=/content/data/models/loras/ \
-e path_embeddings=/content/data/models/embeddings/ \
-e path_vae_approx=/content/data/models/vae_approx/ \
-e path_upscale_models=/content/data/models/upscale_models/ \
-e path_inpaint=/content/data/models/inpaint/ \
-e path_controlnet=/content/data/models/controlnet/ \
-e path_clip_vision=/content/data/models/clip_vision/ \
-e path_fooocus_expansion=/content/data/models/prompt_expansion/fooocus_expansion/ \
-e path_outputs=/content/app/outputs/ \
ghcr.io/lllyasviel/fooocus
```
### Running with Podman
```sh
podman run -p 7865:7865 -v fooocus-data:/content/data -it \
--security-opt=no-new-privileges --cap-drop=ALL --security-opt label=type:nvidia_container_t --device=nvidia.com/gpu=all \
-e CMDARGS=--listen \
-e DATADIR=/content/data \
-e config_path=/content/data/config.txt \
-e config_example_path=/content/data/config_modification_tutorial.txt \
-e path_checkpoints=/content/data/models/checkpoints/ \
-e path_loras=/content/data/models/loras/ \
-e path_embeddings=/content/data/models/embeddings/ \
-e path_vae_approx=/content/data/models/vae_approx/ \
-e path_upscale_models=/content/data/models/upscale_models/ \
-e path_inpaint=/content/data/models/inpaint/ \
-e path_controlnet=/content/data/models/controlnet/ \
-e path_clip_vision=/content/data/models/clip_vision/ \
-e path_fooocus_expansion=/content/data/models/prompt_expansion/fooocus_expansion/ \
-e path_outputs=/content/app/outputs/ \
ghcr.io/lllyasviel/fooocus
```
When you see the message `Use the app with http://0.0.0.0:7865/` in the console, you can access the URL in your browser.
Your models and outputs are stored in the `fooocus-data` volume, which, depending on OS, is stored in `/var/lib/docker/volumes/` (or `~/.local/share/containers/storage/volumes/` when using `podman`).
## Building the container locally
Clone the repository first, and open a terminal in the folder.
Build with `docker`:
```sh
docker build . -t fooocus
```
Build with `podman`:
```sh
podman build . -t fooocus
```
## Details
### Update the container manually (`docker compose`)
When you are using `docker compose up` continuously, the container is not updated to the latest version of Fooocus automatically.
Run `git pull` before executing `docker compose build --no-cache` to build an image with the latest Fooocus version.
You can then start it with `docker compose up`
### Import models, outputs
If you want to import files from models or the outputs folder, you can add the following bind mounts in the [docker-compose.yml](docker-compose.yml) or your preferred method of running the container:
```
#- ./models:/import/models # Once you import files, you don't need to mount again.
#- ./outputs:/import/outputs # Once you import files, you don't need to mount again.
```
After running the container, your files will be copied into `/content/data/models` and `/content/data/outputs`
Since `/content/data` is a persistent volume folder, your files will be persisted even when you re-run the container without the above mounts.
### Paths inside the container
|Path|Details|
|-|-|
|/content/app|The application stored folder|
|/content/app/models.org|Original 'models' folder.<br> Files are copied to the '/content/app/models' which is symlinked to '/content/data/models' every time the container boots. (Existing files will not be overwritten.) |
|/content/data|Persistent volume mount point|
|/content/data/models|The folder is symlinked to '/content/app/models'|
|/content/data/outputs|The folder is symlinked to '/content/app/outputs'|
### Environments
You can change `config.txt` parameters by using environment variables.
**The priority of using the environments is higher than the values defined in `config.txt`, and they will be saved to the `config_modification_tutorial.txt`**
Docker specified environments are there. They are used by 'entrypoint.sh'
|Environment|Details|
|-|-|
|DATADIR|'/content/data' location.|
|CMDARGS|Arguments for [entry_with_update.py](entry_with_update.py) which is called by [entrypoint.sh](entrypoint.sh)|
|config_path|'config.txt' location|
|config_example_path|'config_modification_tutorial.txt' location|
|HF_MIRROR| huggingface mirror site domain|
You can also use the same json key names and values explained in the 'config_modification_tutorial.txt' as the environments.
See examples in the [docker-compose.yml](docker-compose.yml)
## Notes
- Please keep 'path_outputs' under '/content/app'. Otherwise, you may get an error when you open the history log.
- Docker on Mac/Windows still has issues in the form of slow volume access when you use "bind mount" volumes. Please refer to [this article](https://docs.docker.com/storage/volumes/#use-a-volume-with-docker-compose) for not using "bind mount".
- The MPS backend (Metal Performance Shaders, Apple Silicon M1/M2/etc.) is not yet supported in Docker, see https://github.com/pytorch/pytorch/issues/81224
- You can also use `docker compose up -d` to start the container detached and connect to the logs with `docker compose logs -f`. This way you can also close the terminal and keep the container running.

33
entrypoint.sh Executable file
View File

@ -0,0 +1,33 @@
#!/bin/bash
ORIGINALDIR=/content/app
# Use predefined DATADIR if it is defined
[[ x"${DATADIR}" == "x" ]] && DATADIR=/content/data
# Make persistent dir from original dir
function mklink () {
mkdir -p $DATADIR/$1
ln -s $DATADIR/$1 $ORIGINALDIR
}
# Copy old files from import dir
function import () {
(test -d /import/$1 && cd /import/$1 && cp -Rpn . $DATADIR/$1/)
}
cd $ORIGINALDIR
# models
mklink models
# Copy original files
(cd $ORIGINALDIR/models.org && cp -Rpn . $ORIGINALDIR/models/)
# Import old files
import models
# outputs
mklink outputs
# Import old files
import outputs
# Start application
python launch.py $*

60
extras/censor.py Normal file
View File

@ -0,0 +1,60 @@
import os
import numpy as np
import torch
from transformers import CLIPConfig, CLIPImageProcessor
import ldm_patched.modules.model_management as model_management
import modules.config
from extras.safety_checker.models.safety_checker import StableDiffusionSafetyChecker
from ldm_patched.modules.model_patcher import ModelPatcher
safety_checker_repo_root = os.path.join(os.path.dirname(__file__), 'safety_checker')
config_path = os.path.join(safety_checker_repo_root, "configs", "config.json")
preprocessor_config_path = os.path.join(safety_checker_repo_root, "configs", "preprocessor_config.json")
class Censor:
def __init__(self):
self.safety_checker_model: ModelPatcher | None = None
self.clip_image_processor: CLIPImageProcessor | None = None
self.load_device = torch.device('cpu')
self.offload_device = torch.device('cpu')
def init(self):
if self.safety_checker_model is None and self.clip_image_processor is None:
safety_checker_model = modules.config.downloading_safety_checker_model()
self.clip_image_processor = CLIPImageProcessor.from_json_file(preprocessor_config_path)
clip_config = CLIPConfig.from_json_file(config_path)
model = StableDiffusionSafetyChecker.from_pretrained(safety_checker_model, config=clip_config)
model.eval()
self.load_device = model_management.text_encoder_device()
self.offload_device = model_management.text_encoder_offload_device()
model.to(self.offload_device)
self.safety_checker_model = ModelPatcher(model, load_device=self.load_device, offload_device=self.offload_device)
def censor(self, images: list | np.ndarray) -> list | np.ndarray:
self.init()
model_management.load_model_gpu(self.safety_checker_model)
single = False
if not isinstance(images, list) or isinstance(images, np.ndarray):
images = [images]
single = True
safety_checker_input = self.clip_image_processor(images, return_tensors="pt")
safety_checker_input.to(device=self.load_device)
checked_images, has_nsfw_concept = self.safety_checker_model.model(images=images,
clip_input=safety_checker_input.pixel_values)
checked_images = [image.astype(np.uint8) for image in checked_images]
if single:
checked_images = checked_images[0]
return checked_images
default_censor = Censor().censor

3
extras/expansion.py
View File

@ -112,6 +112,9 @@ class FooocusExpansion:
max_token_length = 75 * int(math.ceil(float(current_token_length) / 75.0))
max_new_tokens = max_token_length - current_token_length
if max_new_tokens == 0:
return prompt[:-1]
# https://huggingface.co/blog/introducing-csearch
# https://huggingface.co/docs/transformers/generation_strategies
features = self.model.generate(**tokenized_kwargs,

17
extras/preprocessors.py
View File

@ -1,27 +1,26 @@
import cv2
import numpy as np
import modules.advanced_parameters as advanced_parameters
def centered_canny(x: np.ndarray):
def centered_canny(x: np.ndarray, canny_low_threshold, canny_high_threshold):
assert isinstance(x, np.ndarray)
assert x.ndim == 2 and x.dtype == np.uint8
y = cv2.Canny(x, int(advanced_parameters.canny_low_threshold), int(advanced_parameters.canny_high_threshold))
y = cv2.Canny(x, int(canny_low_threshold), int(canny_high_threshold))
y = y.astype(np.float32) / 255.0
return y
def centered_canny_color(x: np.ndarray):
def centered_canny_color(x: np.ndarray, canny_low_threshold, canny_high_threshold):
assert isinstance(x, np.ndarray)
assert x.ndim == 3 and x.shape[2] == 3
result = [centered_canny(x[..., i]) for i in range(3)]
result = [centered_canny(x[..., i], canny_low_threshold, canny_high_threshold) for i in range(3)]
result = np.stack(result, axis=2)
return result
def pyramid_canny_color(x: np.ndarray):
def pyramid_canny_color(x: np.ndarray, canny_low_threshold, canny_high_threshold):
assert isinstance(x, np.ndarray)
assert x.ndim == 3 and x.shape[2] == 3
@ -31,7 +30,7 @@ def pyramid_canny_color(x: np.ndarray):
for k in [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]:
Hs, Ws = int(H * k), int(W * k)
small = cv2.resize(x, (Ws, Hs), interpolation=cv2.INTER_AREA)
edge = centered_canny_color(small)
edge = centered_canny_color(small, canny_low_threshold, canny_high_threshold)
if acc_edge is None:
acc_edge = edge
else:
@ -54,11 +53,11 @@ def norm255(x, low=4, high=96):
return x * 255.0
def canny_pyramid(x):
def canny_pyramid(x, canny_low_threshold, canny_high_threshold):
# For some reasons, SAI's Control-lora Canny seems to be trained on canny maps with non-standard resolutions.
# Then we use pyramid to use all resolutions to avoid missing any structure in specific resolutions.
color_canny = pyramid_canny_color(x)
color_canny = pyramid_canny_color(x, canny_low_threshold, canny_high_threshold)
result = np.sum(color_canny, axis=2)
return norm255(result, low=1, high=99).clip(0, 255).astype(np.uint8)

171
extras/safety_checker/configs/config.json Normal file
View File

@ -0,0 +1,171 @@
{
"_name_or_path": "clip-vit-large-patch14/",
"architectures": [
"SafetyChecker"
],
"initializer_factor": 1.0,
"logit_scale_init_value": 2.6592,
"model_type": "clip",
"projection_dim": 768,
"text_config": {
"_name_or_path": "",
"add_cross_attention": false,
"architectures": null,
"attention_dropout": 0.0,
"bad_words_ids": null,
"bos_token_id": 0,
"chunk_size_feed_forward": 0,
"cross_attention_hidden_size": null,
"decoder_start_token_id": null,
"diversity_penalty": 0.0,
"do_sample": false,
"dropout": 0.0,
"early_stopping": false,
"encoder_no_repeat_ngram_size": 0,
"eos_token_id": 2,
"exponential_decay_length_penalty": null,
"finetuning_task": null,
"forced_bos_token_id": null,
"forced_eos_token_id": null,
"hidden_act": "quick_gelu",
"hidden_size": 768,
"id2label": {
"0": "LABEL_0",
"1": "LABEL_1"
},
"initializer_factor": 1.0,
"initializer_range": 0.02,
"intermediate_size": 3072,
"is_decoder": false,
"is_encoder_decoder": false,
"label2id": {
"LABEL_0": 0,
"LABEL_1": 1
},
"layer_norm_eps": 1e-05,
"length_penalty": 1.0,
"max_length": 20,
"max_position_embeddings": 77,
"min_length": 0,
"model_type": "clip_text_model",
"no_repeat_ngram_size": 0,
"num_attention_heads": 12,
"num_beam_groups": 1,
"num_beams": 1,
"num_hidden_layers": 12,
"num_return_sequences": 1,
"output_attentions": false,
"output_hidden_states": false,
"output_scores": false,
"pad_token_id": 1,
"prefix": null,
"problem_type": null,
"pruned_heads": {},
"remove_invalid_values": false,
"repetition_penalty": 1.0,
"return_dict": true,
"return_dict_in_generate": false,
"sep_token_id": null,
"task_specific_params": null,
"temperature": 1.0,
"tie_encoder_decoder": false,
"tie_word_embeddings": true,
"tokenizer_class": null,
"top_k": 50,
"top_p": 1.0,
"torch_dtype": null,
"torchscript": false,
"transformers_version": "4.21.0.dev0",
"typical_p": 1.0,
"use_bfloat16": false,
"vocab_size": 49408
},
"text_config_dict": {
"hidden_size": 768,
"intermediate_size": 3072,
"num_attention_heads": 12,
"num_hidden_layers": 12
},
"torch_dtype": "float32",
"transformers_version": null,
"vision_config": {
"_name_or_path": "",
"add_cross_attention": false,
"architectures": null,
"attention_dropout": 0.0,
"bad_words_ids": null,
"bos_token_id": null,
"chunk_size_feed_forward": 0,
"cross_attention_hidden_size": null,
"decoder_start_token_id": null,
"diversity_penalty": 0.0,
"do_sample": false,
"dropout": 0.0,
"early_stopping": false,
"encoder_no_repeat_ngram_size": 0,
"eos_token_id": null,
"exponential_decay_length_penalty": null,
"finetuning_task": null,
"forced_bos_token_id": null,
"forced_eos_token_id": null,
"hidden_act": "quick_gelu",
"hidden_size": 1024,
"id2label": {
"0": "LABEL_0",
"1": "LABEL_1"
},
"image_size": 224,
"initializer_factor": 1.0,
"initializer_range": 0.02,
"intermediate_size": 4096,
"is_decoder": false,
"is_encoder_decoder": false,
"label2id": {
"LABEL_0": 0,
"LABEL_1": 1
},
"layer_norm_eps": 1e-05,
"length_penalty": 1.0,
"max_length": 20,
"min_length": 0,
"model_type": "clip_vision_model",
"no_repeat_ngram_size": 0,
"num_attention_heads": 16,
"num_beam_groups": 1,
"num_beams": 1,
"num_hidden_layers": 24,
"num_return_sequences": 1,
"output_attentions": false,
"output_hidden_states": false,
"output_scores": false,
"pad_token_id": null,
"patch_size": 14,
"prefix": null,
"problem_type": null,
"pruned_heads": {},
"remove_invalid_values": false,
"repetition_penalty": 1.0,
"return_dict": true,
"return_dict_in_generate": false,
"sep_token_id": null,
"task_specific_params": null,
"temperature": 1.0,
"tie_encoder_decoder": false,
"tie_word_embeddings": true,
"tokenizer_class": null,
"top_k": 50,
"top_p": 1.0,
"torch_dtype": null,
"torchscript": false,
"transformers_version": "4.21.0.dev0",
"typical_p": 1.0,
"use_bfloat16": false
},
"vision_config_dict": {
"hidden_size": 1024,
"intermediate_size": 4096,
"num_attention_heads": 16,
"num_hidden_layers": 24,
"patch_size": 14
}
}

20
extras/safety_checker/configs/preprocessor_config.json Normal file
View File

@ -0,0 +1,20 @@
{
"crop_size": 224,
"do_center_crop": true,
"do_convert_rgb": true,
"do_normalize": true,
"do_resize": true,
"feature_extractor_type": "CLIPFeatureExtractor",
"image_mean": [
0.48145466,
0.4578275,
0.40821073
],
"image_std": [
0.26862954,
0.26130258,
0.27577711
],
"resample": 3,
"size": 224
}

126
extras/safety_checker/models/safety_checker.py Normal file
View File

@ -0,0 +1,126 @@
# from https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/safety_checker.py
# Copyright 2024 The HuggingFace Team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import numpy as np
import torch
import torch.nn as nn
from transformers import CLIPConfig, CLIPVisionModel, PreTrainedModel
from transformers.utils import logging
logger = logging.get_logger(__name__)
def cosine_distance(image_embeds, text_embeds):
normalized_image_embeds = nn.functional.normalize(image_embeds)
normalized_text_embeds = nn.functional.normalize(text_embeds)
return torch.mm(normalized_image_embeds, normalized_text_embeds.t())
class StableDiffusionSafetyChecker(PreTrainedModel):
config_class = CLIPConfig
main_input_name = "clip_input"
_no_split_modules = ["CLIPEncoderLayer"]
def __init__(self, config: CLIPConfig):
super().__init__(config)
self.vision_model = CLIPVisionModel(config.vision_config)
self.visual_projection = nn.Linear(config.vision_config.hidden_size, config.projection_dim, bias=False)
self.concept_embeds = nn.Parameter(torch.ones(17, config.projection_dim), requires_grad=False)
self.special_care_embeds = nn.Parameter(torch.ones(3, config.projection_dim), requires_grad=False)
self.concept_embeds_weights = nn.Parameter(torch.ones(17), requires_grad=False)
self.special_care_embeds_weights = nn.Parameter(torch.ones(3), requires_grad=False)
@torch.no_grad()
def forward(self, clip_input, images):
pooled_output = self.vision_model(clip_input)[1] # pooled_output
image_embeds = self.visual_projection(pooled_output)
# we always cast to float32 as this does not cause significant overhead and is compatible with bfloat16
special_cos_dist = cosine_distance(image_embeds, self.special_care_embeds).cpu().float().numpy()
cos_dist = cosine_distance(image_embeds, self.concept_embeds).cpu().float().numpy()
result = []
batch_size = image_embeds.shape[0]
for i in range(batch_size):
result_img = {"special_scores": {}, "special_care": [], "concept_scores": {}, "bad_concepts": []}
# increase this value to create a stronger `nfsw` filter
# at the cost of increasing the possibility of filtering benign images
adjustment = 0.0
for concept_idx in range(len(special_cos_dist[0])):
concept_cos = special_cos_dist[i][concept_idx]
concept_threshold = self.special_care_embeds_weights[concept_idx].item()
result_img["special_scores"][concept_idx] = round(concept_cos - concept_threshold + adjustment, 3)
if result_img["special_scores"][concept_idx] > 0:
result_img["special_care"].append({concept_idx, result_img["special_scores"][concept_idx]})
adjustment = 0.01
for concept_idx in range(len(cos_dist[0])):
concept_cos = cos_dist[i][concept_idx]
concept_threshold = self.concept_embeds_weights[concept_idx].item()
result_img["concept_scores"][concept_idx] = round(concept_cos - concept_threshold + adjustment, 3)
if result_img["concept_scores"][concept_idx] > 0:
result_img["bad_concepts"].append(concept_idx)
result.append(result_img)
has_nsfw_concepts = [len(res["bad_concepts"]) > 0 for res in result]
for idx, has_nsfw_concept in enumerate(has_nsfw_concepts):
if has_nsfw_concept:
if torch.is_tensor(images) or torch.is_tensor(images[0]):
images[idx] = torch.zeros_like(images[idx]) # black image
else:
images[idx] = np.zeros(images[idx].shape) # black image
if any(has_nsfw_concepts):
logger.warning(
"Potential NSFW content was detected in one or more images. A black image will be returned instead."
" Try again with a different prompt and/or seed."
)
return images, has_nsfw_concepts
@torch.no_grad()
def forward_onnx(self, clip_input: torch.Tensor, images: torch.Tensor):
pooled_output = self.vision_model(clip_input)[1] # pooled_output
image_embeds = self.visual_projection(pooled_output)
special_cos_dist = cosine_distance(image_embeds, self.special_care_embeds)
cos_dist = cosine_distance(image_embeds, self.concept_embeds)
# increase this value to create a stronger `nsfw` filter
# at the cost of increasing the possibility of filtering benign images
adjustment = 0.0
special_scores = special_cos_dist - self.special_care_embeds_weights + adjustment
# special_scores = special_scores.round(decimals=3)
special_care = torch.any(special_scores > 0, dim=1)
special_adjustment = special_care * 0.01
special_adjustment = special_adjustment.unsqueeze(1).expand(-1, cos_dist.shape[1])
concept_scores = (cos_dist - self.concept_embeds_weights) + special_adjustment
# concept_scores = concept_scores.round(decimals=3)
has_nsfw_concepts = torch.any(concept_scores > 0, dim=1)
images[has_nsfw_concepts] = 0.0 # black image
return images, has_nsfw_concepts

98
extras/vae_interpose.py
View File

@ -1,69 +1,85 @@
# https://github.com/city96/SD-Latent-Interposer/blob/main/interposer.py
import os
import torch
import safetensors.torch as sf
import torch.nn as nn
import ldm_patched.modules.model_management
import safetensors.torch as sf
import torch
import torch.nn as nn
import ldm_patched.modules.model_management
from ldm_patched.modules.model_patcher import ModelPatcher
from modules.config import path_vae_approx
class Block(nn.Module):
def __init__(self, size):
class ResBlock(nn.Module):
"""Block with residuals"""
def __init__(self, ch):
super().__init__()
self.join = nn.ReLU()
self.norm = nn.BatchNorm2d(ch)
self.long = nn.Sequential(
nn.Conv2d(size, size, kernel_size=3, stride=1, padding=1),
nn.LeakyReLU(0.1),
nn.Conv2d(size, size, kernel_size=3, stride=1, padding=1),
nn.LeakyReLU(0.1),
nn.Conv2d(size, size, kernel_size=3, stride=1, padding=1),
nn.Conv2d(ch, ch, kernel_size=3, stride=1, padding=1),
nn.SiLU(),
nn.Conv2d(ch, ch, kernel_size=3, stride=1, padding=1),
nn.SiLU(),
nn.Conv2d(ch, ch, kernel_size=3, stride=1, padding=1),
nn.Dropout(0.1)
)
def forward(self, x):
y = self.long(x)
z = self.join(y + x)
return z
x = self.norm(x)
return self.join(self.long(x) + x)
class Interposer(nn.Module):
def __init__(self):
class ExtractBlock(nn.Module):
"""Increase no. of channels by [out/in]"""
def __init__(self, ch_in, ch_out):
super().__init__()
self.chan = 4
self.hid = 128
self.head_join = nn.ReLU()
self.head_short = nn.Conv2d(self.chan, self.hid, kernel_size=3, stride=1, padding=1)
self.head_long = nn.Sequential(
nn.Conv2d(self.chan, self.hid, kernel_size=3, stride=1, padding=1),
nn.LeakyReLU(0.1),
nn.Conv2d(self.hid, self.hid, kernel_size=3, stride=1, padding=1),
nn.LeakyReLU(0.1),
nn.Conv2d(self.hid, self.hid, kernel_size=3, stride=1, padding=1),
)
self.core = nn.Sequential(
Block(self.hid),
Block(self.hid),
Block(self.hid),
)
self.tail = nn.Sequential(
nn.ReLU(),
nn.Conv2d(self.hid, self.chan, kernel_size=3, stride=1, padding=1)
self.join = nn.ReLU()
self.short = nn.Conv2d(ch_in, ch_out, kernel_size=3, stride=1, padding=1)
self.long = nn.Sequential(
nn.Conv2d(ch_in, ch_out, kernel_size=3, stride=1, padding=1),
nn.SiLU(),
nn.Conv2d(ch_out, ch_out, kernel_size=3, stride=1, padding=1),
nn.SiLU(),
nn.Conv2d(ch_out, ch_out, kernel_size=3, stride=1, padding=1),
nn.Dropout(0.1)
)
def forward(self, x):
y = self.head_join(
self.head_long(x) +
self.head_short(x)
return self.join(self.long(x) + self.short(x))
class InterposerModel(nn.Module):
"""Main neural network"""
def __init__(self, ch_in=4, ch_out=4, ch_mid=64, scale=1.0, blocks=12):
super().__init__()
self.ch_in = ch_in
self.ch_out = ch_out
self.ch_mid = ch_mid
self.blocks = blocks
self.scale = scale
self.head = ExtractBlock(self.ch_in, self.ch_mid)
self.core = nn.Sequential(
nn.Upsample(scale_factor=self.scale, mode="nearest"),
*[ResBlock(self.ch_mid) for _ in range(blocks)],
nn.BatchNorm2d(self.ch_mid),
nn.SiLU(),
)
self.tail = nn.Conv2d(self.ch_mid, self.ch_out, kernel_size=3, stride=1, padding=1)
def forward(self, x):
y = self.head(x)
z = self.core(y)
return self.tail(z)
vae_approx_model = None
vae_approx_filename = os.path.join(path_vae_approx, 'xl-to-v1_interposer-v3.1.safetensors')
vae_approx_filename = os.path.join(path_vae_approx, 'xl-to-v1_interposer-v4.0.safetensors')
def parse(x):
@ -72,7 +88,7 @@ def parse(x):
x_origin = x.clone()
if vae_approx_model is None:
model = Interposer()
model = InterposerModel()
model.eval()
sd = sf.load_file(vae_approx_filename)
model.load_state_dict(sd)

2
fooocus_colab.ipynb
View File

@ -12,7 +12,7 @@
"%cd /content\n",
"!git clone https://github.com/lllyasviel/Fooocus.git\n",
"%cd /content/Fooocus\n",
"!python entry_with_update.py --share\n"
"!python entry_with_update.py --share --always-high-vram\n"
]
}
],

2
fooocus_version.py
View File

@ -1 +1 @@
version = '2.1.864'
version = '2.4.0'

4
javascript/contextMenus.js
View File

@ -154,12 +154,8 @@ let cancelGenerateForever = function() {
let generateOnRepeatForButtons = function() {
generateOnRepeat('#generate_button', '#stop_button');
};
appendContextMenuOption('#generate_button', 'Generate forever', generateOnRepeatForButtons);
// appendContextMenuOption('#stop_button', 'Generate forever', generateOnRepeatForButtons);
// appendContextMenuOption('#stop_button', 'Cancel generate forever', cancelGenerateForever);
// appendContextMenuOption('#generate_button', 'Cancel generate forever', cancelGenerateForever);
})();
//End example Context Menu Items

6
javascript/localization.js
View File

@ -80,6 +80,12 @@ function refresh_style_localization() {
processNode(document.querySelector('.style_selections'));
}
function refresh_aspect_ratios_label(value) {
label = document.querySelector('#aspect_ratios_accordion div span[data-original-text="Aspect Ratios"]')
translation = getTranslation("Aspect Ratios")
label.textContent = translation + " " + htmlDecode(value)
}
function localizeWholePage() {
processNode(gradioApp());

58
javascript/script.js
View File

@ -122,6 +122,43 @@ document.addEventListener("DOMContentLoaded", function() {
initStylePreviewOverlay();
});
var onAppend = function(elem, f) {
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
if (m.addedNodes.length) {
f(m.addedNodes);
}
});
});
observer.observe(elem, {childList: true});
}
function addObserverIfDesiredNodeAvailable(querySelector, callback) {
var elem = document.querySelector(querySelector);
if (!elem) {
window.setTimeout(() => addObserverIfDesiredNodeAvailable(querySelector, callback), 1000);
return;
}
onAppend(elem, callback);
}
/**
* Show reset button on toast "Connection errored out."
*/
addObserverIfDesiredNodeAvailable(".toast-wrap", function(added) {
added.forEach(function(element) {
if (element.innerText.includes("Connection errored out.")) {
window.setTimeout(function() {
document.getElementById("reset_button").classList.remove("hidden");
document.getElementById("generate_button").classList.add("hidden");
document.getElementById("skip_button").classList.add("hidden");
document.getElementById("stop_button").classList.add("hidden");
});
}
});
});
/**
* Add a ctrl+enter as a shortcut to start a generation
*/
@ -150,9 +187,12 @@ function initStylePreviewOverlay() {
let overlayVisible = false;
const samplesPath = document.querySelector("meta[name='samples-path']").getAttribute("content")
const overlay = document.createElement('div');
const tooltip = document.createElement('div');
tooltip.className = 'preview-tooltip';
overlay.appendChild(tooltip);
overlay.id = 'stylePreviewOverlay';
document.body.appendChild(overlay);
document.addEventListener('mouseover', function(e) {
document.addEventListener('mouseover', function (e) {
const label = e.target.closest('.style_selections label');
if (!label) return;
label.removeEventListener("mouseout", onMouseLeave);
@ -162,9 +202,12 @@ function initStylePreviewOverlay() {
const originalText = label.querySelector("span").getAttribute("data-original-text");
const name = originalText || label.querySelector("span").textContent;
overlay.style.backgroundImage = `url("${samplesPath.replace(
"fooocus_v2",
name.toLowerCase().replaceAll(" ", "_")
"fooocus_v2",
name.toLowerCase().replaceAll(" ", "_")
).replaceAll("\\", "\\\\")}")`;
tooltip.textContent = name;
function onMouseLeave() {
overlayVisible = false;
overlay.style.opacity = "0";
@ -172,8 +215,8 @@ function initStylePreviewOverlay() {
label.removeEventListener("mouseout", onMouseLeave);
}
});
document.addEventListener('mousemove', function(e) {
if(!overlayVisible) return;
document.addEventListener('mousemove', function (e) {
if (!overlayVisible) return;
overlay.style.left = `${e.clientX}px`;
overlay.style.top = `${e.clientY}px`;
overlay.className = e.clientY > window.innerHeight / 2 ? "lower-half" : "upper-half";
@ -213,3 +256,8 @@ function set_theme(theme) {
window.location.replace(gradioURL + '?__theme=' + theme);
}
}
function htmlDecode(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}

68
launch.py
View File

@ -1,6 +1,6 @@
import os
import sys
import ssl
import sys
print('[System ARGV] ' + str(sys.argv))
@ -10,19 +10,17 @@ os.chdir(root)
os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
os.environ["PYTORCH_MPS_HIGH_WATERMARK_RATIO"] = "0.0"
os.environ["GRADIO_SERVER_PORT"] = "7865"
if "GRADIO_SERVER_PORT" not in os.environ:
os.environ["GRADIO_SERVER_PORT"] = "7865"
ssl._create_default_https_context = ssl._create_unverified_context
import platform
import fooocus_version
from build_launcher import build_launcher
from modules.launch_util import is_installed, run, python, run_pip, requirements_met
from modules.launch_util import is_installed, run, python, run_pip, requirements_met, delete_folder_content
from modules.model_loader import load_file_from_url
from modules import config
REINSTALL_ALL = False
TRY_INSTALL_XFORMERS = False
@ -42,7 +40,7 @@ def prepare_environment():
if TRY_INSTALL_XFORMERS:
if REINSTALL_ALL or not is_installed("xformers"):
xformers_package = os.environ.get('XFORMERS_PACKAGE', 'xformers==0.0.20')
xformers_package = os.environ.get('XFORMERS_PACKAGE', 'xformers==0.0.23')
if platform.system() == "Windows":
if platform.python_version().startswith("3.10"):
run_pip(f"install -U -I --no-deps {xformers_package}", "xformers", live=True)
@ -64,8 +62,8 @@ def prepare_environment():
vae_approx_filenames = [
('xlvaeapp.pth', 'https://huggingface.co/lllyasviel/misc/resolve/main/xlvaeapp.pth'),
('vaeapp_sd15.pth', 'https://huggingface.co/lllyasviel/misc/resolve/main/vaeapp_sd15.pt'),
('xl-to-v1_interposer-v3.1.safetensors',
'https://huggingface.co/lllyasviel/misc/resolve/main/xl-to-v1_interposer-v3.1.safetensors')
('xl-to-v1_interposer-v4.0.safetensors',
'https://huggingface.co/mashb1t/misc/resolve/main/xl-to-v1_interposer-v4.0.safetensors')
]
@ -78,13 +76,28 @@ prepare_environment()
build_launcher()
args = ini_args()
if args.gpu_device_id is not None:
os.environ['CUDA_VISIBLE_DEVICES'] = str(args.gpu_device_id)
print("Set device to:", args.gpu_device_id)
if args.hf_mirror is not None :
os.environ['HF_MIRROR'] = str(args.hf_mirror)
print("Set hf_mirror to:", args.hf_mirror)
def download_models():
from modules import config
os.environ['GRADIO_TEMP_DIR'] = config.temp_path
if config.temp_path_cleanup_on_launch:
print(f'[Cleanup] Attempting to delete content of temp dir {config.temp_path}')
result = delete_folder_content(config.temp_path, '[Cleanup] ')
if result:
print("[Cleanup] Cleanup successful")
else:
print(f"[Cleanup] Failed to delete content of temp dir.")
def download_models(default_model, previous_default_models, checkpoint_downloads, embeddings_downloads, lora_downloads):
for file_name, url in vae_approx_filenames:
load_file_from_url(url=url, model_dir=config.path_vae_approx, file_name=file_name)
@ -96,31 +109,32 @@ def download_models():
if args.disable_preset_download:
print('Skipped model download.')
return
return default_model, checkpoint_downloads
if not args.always_download_new_model:
if not os.path.exists(os.path.join(config.path_checkpoints, config.default_base_model_name)):
for alternative_model_name in config.previous_default_models:
if os.path.exists(os.path.join(config.path_checkpoints, alternative_model_name)):
print(f'You do not have [{config.default_base_model_name}] but you have [{alternative_model_name}].')
if not os.path.exists(os.path.join(config.paths_checkpoints[0], default_model)):
for alternative_model_name in previous_default_models:
if os.path.exists(os.path.join(config.paths_checkpoints[0], alternative_model_name)):
print(f'You do not have [{default_model}] but you have [{alternative_model_name}].')
print(f'Fooocus will use [{alternative_model_name}] to avoid downloading new models, '
f'but you are not using latest models.')
f'but you are not using the latest models.')
print('Use --always-download-new-model to avoid fallback and always get new models.')
config.checkpoint_downloads = {}
config.default_base_model_name = alternative_model_name
checkpoint_downloads = {}
default_model = alternative_model_name
break
for file_name, url in config.checkpoint_downloads.items():
load_file_from_url(url=url, model_dir=config.path_checkpoints, file_name=file_name)
for file_name, url in config.embeddings_downloads.items():
for file_name, url in checkpoint_downloads.items():
load_file_from_url(url=url, model_dir=config.paths_checkpoints[0], file_name=file_name)
for file_name, url in embeddings_downloads.items():
load_file_from_url(url=url, model_dir=config.path_embeddings, file_name=file_name)
for file_name, url in config.lora_downloads.items():
load_file_from_url(url=url, model_dir=config.path_loras, file_name=file_name)
for file_name, url in lora_downloads.items():
load_file_from_url(url=url, model_dir=config.paths_loras[0], file_name=file_name)
return
return default_model, checkpoint_downloads
download_models()
config.default_base_model_name, config.checkpoint_downloads = download_models(
config.default_base_model_name, config.previous_default_models, config.checkpoint_downloads,
config.embeddings_downloads, config.lora_downloads)
from webui import *

55
ldm_patched/contrib/external_align_your_steps.py Normal file
View File

@ -0,0 +1,55 @@
# https://github.com/comfyanonymous/ComfyUI/blob/master/nodes.py
#from: https://research.nvidia.com/labs/toronto-ai/AlignYourSteps/howto.html
import numpy as np
import torch
def loglinear_interp(t_steps, num_steps):
"""
Performs log-linear interpolation of a given array of decreasing numbers.
"""
xs = np.linspace(0, 1, len(t_steps))
ys = np.log(t_steps[::-1])
new_xs = np.linspace(0, 1, num_steps)
new_ys = np.interp(new_xs, xs, ys)
interped_ys = np.exp(new_ys)[::-1].copy()
return interped_ys
NOISE_LEVELS = {"SD1": [14.6146412293, 6.4745760956, 3.8636745985, 2.6946151520, 1.8841921177, 1.3943805092, 0.9642583904, 0.6523686016, 0.3977456272, 0.1515232662, 0.0291671582],
"SDXL":[14.6146412293, 6.3184485287, 3.7681790315, 2.1811480769, 1.3405244945, 0.8620721141, 0.5550693289, 0.3798540708, 0.2332364134, 0.1114188177, 0.0291671582],
"SVD": [700.00, 54.5, 15.886, 7.977, 4.248, 1.789, 0.981, 0.403, 0.173, 0.034, 0.002]}
class AlignYourStepsScheduler:
@classmethod
def INPUT_TYPES(s):
return {"required":
{"model_type": (["SD1", "SDXL", "SVD"], ),
"steps": ("INT", {"default": 10, "min": 10, "max": 10000}),
"denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
}
}
RETURN_TYPES = ("SIGMAS",)
CATEGORY = "sampling/custom_sampling/schedulers"
FUNCTION = "get_sigmas"
def get_sigmas(self, model_type, steps, denoise):
total_steps = steps
if denoise < 1.0:
if denoise <= 0.0:
return (torch.FloatTensor([]),)
total_steps = round(steps * denoise)
sigmas = NOISE_LEVELS[model_type][:]
if (steps + 1) != len(sigmas):
sigmas = loglinear_interp(sigmas, steps + 1)
sigmas = sigmas[-(total_steps + 1):]
sigmas[-1] = 0
return (torch.FloatTensor(sigmas), )
NODE_CLASS_MAPPINGS = {
"AlignYourStepsScheduler": AlignYourStepsScheduler,
}

6
ldm_patched/contrib/external_canny.py
View File

@ -78,7 +78,7 @@ def spatial_gradient(input, normalized: bool = True):
Return:
the derivatives of the input feature map. with shape :math:`(B, C, 2, H, W)`.
.. note::
See a working example `here <https://kornia-tutorials.readthedocs.io/en/latest/
See a working example `here <https://kornia.readthedocs.io/en/latest/
filtering_edges.html>`__.
Examples:
>>> input = torch.rand(1, 3, 4, 4)
@ -120,7 +120,7 @@ def rgb_to_grayscale(image, rgb_weights = None):
grayscale version of the image with shape :math:`(*,1,H,W)`.
.. note::
See a working example `here <https://kornia-tutorials.readthedocs.io/en/latest/
See a working example `here <https://kornia.readthedocs.io/en/latest/
color_conversions.html>`__.
Example:
@ -176,7 +176,7 @@ def canny(
- the canny edge magnitudes map, shape of :math:`(B,1,H,W)`.
- the canny edge detection filtered by thresholds and hysteresis, shape of :math:`(B,1,H,W)`.
.. note::
See a working example `here <https://kornia-tutorials.readthedocs.io/en/latest/
See a working example `here <https://kornia.readthedocs.io/en/latest/
canny.html>`__.
Example:
>>> input = torch.rand(5, 3, 4, 4)

20
ldm_patched/contrib/external_custom_sampler.py
View File

@ -230,6 +230,25 @@ class SamplerDPMPP_SDE:
sampler = ldm_patched.modules.samplers.ksampler(sampler_name, {"eta": eta, "s_noise": s_noise, "r": r})
return (sampler, )
class SamplerTCD:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"eta": ("FLOAT", {"default": 0.3, "min": 0.0, "max": 1.0, "step": 0.01}),
}
}
RETURN_TYPES = ("SAMPLER",)
CATEGORY = "sampling/custom_sampling/samplers"
FUNCTION = "get_sampler"
def get_sampler(self, eta=0.3):
sampler = ldm_patched.modules.samplers.ksampler("tcd", {"eta": eta})
return (sampler, )
class SamplerCustom:
@classmethod
def INPUT_TYPES(s):
@ -292,6 +311,7 @@ NODE_CLASS_MAPPINGS = {
"KSamplerSelect": KSamplerSelect,
"SamplerDPMPP_2M_SDE": SamplerDPMPP_2M_SDE,
"SamplerDPMPP_SDE": SamplerDPMPP_SDE,
"SamplerTCD": SamplerTCD,
"SplitSigmas": SplitSigmas,
"FlipSigmas": FlipSigmas,
}

5
ldm_patched/contrib/external_model_advanced.py
View File

@ -70,7 +70,7 @@ class ModelSamplingDiscrete:
@classmethod
def INPUT_TYPES(s):
return {"required": { "model": ("MODEL",),
"sampling": (["eps", "v_prediction", "lcm"],),
"sampling": (["eps", "v_prediction", "lcm", "tcd"]),
"zsnr": ("BOOLEAN", {"default": False}),
}}
@ -90,6 +90,9 @@ class ModelSamplingDiscrete:
elif sampling == "lcm":
sampling_type = LCM
sampling_base = ModelSamplingDiscreteDistilled
elif sampling == "tcd":
sampling_type = ldm_patched.modules.model_sampling.EPS
sampling_base = ModelSamplingDiscreteDistilled
class ModelSamplingAdvanced(sampling_base, sampling_type):
pass

28
ldm_patched/k_diffusion/sampling.py
View File

@ -752,7 +752,6 @@ def sample_lcm(model, x, sigmas, extra_args=None, callback=None, disable=None, n
return x
@torch.no_grad()
def sample_heunpp2(model, x, sigmas, extra_args=None, callback=None, disable=None, s_churn=0., s_tmin=0., s_tmax=float('inf'), s_noise=1.):
# From MIT licensed: https://github.com/Carzit/sd-webui-samplers-scheduler/
@ -808,3 +807,30 @@ def sample_heunpp2(model, x, sigmas, extra_args=None, callback=None, disable=Non
d_prime = w1 * d + w2 * d_2 + w3 * d_3
x = x + d_prime * dt
return x
@torch.no_grad()
def sample_tcd(model, x, sigmas, extra_args=None, callback=None, disable=None, noise_sampler=None, eta=0.3):
extra_args = {} if extra_args is None else extra_args
noise_sampler = default_noise_sampler(x) if noise_sampler is None else noise_sampler
s_in = x.new_ones([x.shape[0]])
model_sampling = model.inner_model.inner_model.model_sampling
timesteps_s = torch.floor((1 - eta) * model_sampling.timestep(sigmas)).to(dtype=torch.long).detach().cpu()
timesteps_s[-1] = 0
alpha_prod_s = model_sampling.alphas_cumprod[timesteps_s]
beta_prod_s = 1 - alpha_prod_s
for i in trange(len(sigmas) - 1, disable=disable):
denoised = model(x, sigmas[i] * s_in, **extra_args) # predicted_original_sample
eps = (x - denoised) / sigmas[i]
denoised = alpha_prod_s[i + 1].sqrt() * denoised + beta_prod_s[i + 1].sqrt() * eps
if callback is not None:
callback({"x": x, "i": i, "sigma": sigmas[i], "sigma_hat": sigmas[i], "denoised": denoised})
x = denoised
if eta > 0 and sigmas[i + 1] > 0:
noise = noise_sampler(sigmas[i], sigmas[i + 1])
x = x / alpha_prod_s[i+1].sqrt() + noise * (sigmas[i+1]**2 + 1 - 1/alpha_prod_s[i+1]).sqrt()
return x

4
ldm_patched/modules/args_parser.py
View File

@ -37,6 +37,7 @@ parser.add_argument("--listen", type=str, default="127.0.0.1", metavar="IP", nar
parser.add_argument("--port", type=int, default=8188)
parser.add_argument("--disable-header-check", type=str, default=None, metavar="ORIGIN", nargs="?", const="*")
parser.add_argument("--web-upload-size", type=float, default=100)
parser.add_argument("--hf-mirror", type=str, default=None)
parser.add_argument("--external-working-path", type=str, default=None, metavar="PATH", nargs='+', action='append')
parser.add_argument("--output-path", type=str, default=None)
@ -100,8 +101,7 @@ vram_group.add_argument("--always-high-vram", action="store_true")
vram_group.add_argument("--always-normal-vram", action="store_true")
vram_group.add_argument("--always-low-vram", action="store_true")
vram_group.add_argument("--always-no-vram", action="store_true")
vram_group.add_argument("--always-cpu", action="store_true")
vram_group.add_argument("--always-cpu", type=int, nargs="?", metavar="CPU_NUM_THREADS", const=-1)
parser.add_argument("--always-offload-from-vram", action="store_true")
parser.add_argument("--pytorch-deterministic", action="store_true")

6
ldm_patched/modules/conds.py
View File

@ -3,8 +3,6 @@ import math
import ldm_patched.modules.utils
def lcm(a, b): #TODO: eventually replace by math.lcm (added in python3.9)
return abs(a*b) // math.gcd(a, b)
class CONDRegular:
def __init__(self, cond):
@ -41,7 +39,7 @@ class CONDCrossAttn(CONDRegular):
if s1[0] != s2[0] or s1[2] != s2[2]: #these 2 cases should not happen
return False
mult_min = lcm(s1[1], s2[1])
mult_min = math.lcm(s1[1], s2[1])
diff = mult_min // min(s1[1], s2[1])
if diff > 4: #arbitrary limit on the padding because it's probably going to impact performance negatively if it's too much
return False
@ -52,7 +50,7 @@ class CONDCrossAttn(CONDRegular):
crossattn_max_len = self.cond.shape[1]
for x in others:
c = x.cond
crossattn_max_len = lcm(crossattn_max_len, c.shape[1])
crossattn_max_len = math.lcm(crossattn_max_len, c.shape[1])
conds.append(c)
out = []

3
ldm_patched/modules/model_management.py
View File

@ -60,6 +60,9 @@ except:
pass
if args.always_cpu:
if args.always_cpu > 0:
torch.set_num_threads(args.always_cpu)
print(f"Running on {torch.get_num_threads()} CPU threads")
cpu_state = CPUState.CPU
def is_intel_xpu():

8
ldm_patched/modules/model_sampling.py
View File

@ -50,17 +50,17 @@ class ModelSamplingDiscrete(torch.nn.Module):
self.linear_start = linear_start
self.linear_end = linear_end
# self.register_buffer('betas', torch.tensor(betas, dtype=torch.float32))
# self.register_buffer('alphas_cumprod', torch.tensor(alphas_cumprod, dtype=torch.float32))
# self.register_buffer('alphas_cumprod_prev', torch.tensor(alphas_cumprod_prev, dtype=torch.float32))
sigmas = ((1 - alphas_cumprod) / alphas_cumprod) ** 0.5
self.set_sigmas(sigmas)
self.set_alphas_cumprod(alphas_cumprod.float())
def set_sigmas(self, sigmas):
self.register_buffer('sigmas', sigmas)
self.register_buffer('log_sigmas', sigmas.log())
def set_alphas_cumprod(self, alphas_cumprod):
self.register_buffer("alphas_cumprod", alphas_cumprod.float())
@property
def sigma_min(self):
return self.sigmas[0]

2
ldm_patched/modules/samplers.py
View File

@ -523,7 +523,7 @@ class UNIPCBH2(Sampler):
KSAMPLER_NAMES = ["euler", "euler_ancestral", "heun", "heunpp2","dpm_2", "dpm_2_ancestral",
"lms", "dpm_fast", "dpm_adaptive", "dpmpp_2s_ancestral", "dpmpp_sde", "dpmpp_sde_gpu",
"dpmpp_2m", "dpmpp_2m_sde", "dpmpp_2m_sde_gpu", "dpmpp_3m_sde", "dpmpp_3m_sde_gpu", "ddpm", "lcm"]
"dpmpp_2m", "dpmpp_2m_sde", "dpmpp_2m_sde_gpu", "dpmpp_3m_sde", "dpmpp_3m_sde_gpu", "ddpm", "lcm", "tcd"]
class KSAMPLER(Sampler):
def __init__(self, sampler_function, extra_options={}, inpaint_options={}):

13
ldm_patched/modules/sd.py
View File

@ -427,12 +427,13 @@ def load_checkpoint(config_path=None, ckpt_path=None, output_vae=True, output_cl
return (ldm_patched.modules.model_patcher.ModelPatcher(model, load_device=model_management.get_torch_device(), offload_device=offload_device), clip, vae)
def load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, output_clipvision=False, embedding_directory=None, output_model=True):
def load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, output_clipvision=False, embedding_directory=None, output_model=True, vae_filename_param=None):
sd = ldm_patched.modules.utils.load_torch_file(ckpt_path)
sd_keys = sd.keys()
clip = None
clipvision = None
vae = None
vae_filename = None
model = None
model_patcher = None
clip_target = None
@ -462,8 +463,12 @@ def load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, o
model.load_model_weights(sd, "model.diffusion_model.")
if output_vae:
vae_sd = ldm_patched.modules.utils.state_dict_prefix_replace(sd, {"first_stage_model.": ""}, filter_keys=True)
vae_sd = model_config.process_vae_state_dict(vae_sd)
if vae_filename_param is None:
vae_sd = ldm_patched.modules.utils.state_dict_prefix_replace(sd, {"first_stage_model.": ""}, filter_keys=True)
vae_sd = model_config.process_vae_state_dict(vae_sd)
else:
vae_sd = ldm_patched.modules.utils.load_torch_file(vae_filename_param)
vae_filename = vae_filename_param
vae = VAE(sd=vae_sd)
if output_clip:
@ -485,7 +490,7 @@ def load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, o
print("loaded straight to GPU")
model_management.load_model_gpu(model_patcher)
return (model_patcher, clip, vae, clipvision)
return model_patcher, clip, vae, vae_filename, clipvision
def load_unet_state_dict(sd): #load unet in diffusers format

4
ldm_patched/pfn/architecture/HAT.py
View File

@ -14,7 +14,7 @@ from .timm.weight_init import trunc_normal_
def drop_path(x, drop_prob: float = 0.0, training: bool = False):
"""Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).
From: https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/layers/drop.py
From: https://github.com/huggingface/pytorch-image-models/blob/main/timm/layers/drop.py
"""
if drop_prob == 0.0 or not training:
return x
@ -30,7 +30,7 @@ def drop_path(x, drop_prob: float = 0.0, training: bool = False):
class DropPath(nn.Module):
"""Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).
From: https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/layers/drop.py
From: https://github.com/huggingface/pytorch-image-models/blob/main/timm/layers/drop.py
"""
def __init__(self, drop_prob=None):

2
ldm_patched/pfn/architecture/RRDB.py
View File

@ -13,7 +13,7 @@ import torch.nn.functional as F
from . import block as B
# Borrowed from https://github.com/rlaphoenix/VSGAN/blob/master/vsgan/archs/ESRGAN.py
# Borrowed from https://github.com/rlaphoenix/VSGAN/blob/master/vsgan/archs/esrgan.py
# Which enhanced stuff that was already here
class RRDBNet(nn.Module):
def __init__(

2
ldm_patched/pfn/architecture/face/codeformer.py
View File

@ -2,7 +2,7 @@
Modified from https://github.com/sczhou/CodeFormer
VQGAN code, adapted from the original created by the Unleashing Transformers authors:
https://github.com/samb-t/unleashing-transformers/blob/master/models/vqgan.py
This verison of the arch specifically was gathered from an old version of GFPGAN. If this is a problem, please contact me.
This version of the arch specifically was gathered from an old version of GFPGAN. If this is a problem, please contact me.
"""
import math
from typing import Optional

0
models/safety_checker/put_safety_checker_models_here Normal file
View File

0
modules/__init__.py Normal file
View File

33
modules/advanced_parameters.py
View File

@ -1,33 +0,0 @@
disable_preview, adm_scaler_positive, adm_scaler_negative, adm_scaler_end, adaptive_cfg, sampler_name, \
scheduler_name, generate_image_grid, overwrite_step, overwrite_switch, overwrite_width, overwrite_height, \
overwrite_vary_strength, overwrite_upscale_strength, \
mixing_image_prompt_and_vary_upscale, mixing_image_prompt_and_inpaint, \
debugging_cn_preprocessor, skipping_cn_preprocessor, controlnet_softness, canny_low_threshold, canny_high_threshold, \
refiner_swap_method, \
freeu_enabled, freeu_b1, freeu_b2, freeu_s1, freeu_s2, \
debugging_inpaint_preprocessor, inpaint_disable_initial_latent, inpaint_engine, inpaint_strength, inpaint_respective_field, \
inpaint_mask_upload_checkbox, invert_mask_checkbox, inpaint_erode_or_dilate = [None] * 35
def set_all_advanced_parameters(*args):
global disable_preview, adm_scaler_positive, adm_scaler_negative, adm_scaler_end, adaptive_cfg, sampler_name, \
scheduler_name, generate_image_grid, overwrite_step, overwrite_switch, overwrite_width, overwrite_height, \
overwrite_vary_strength, overwrite_upscale_strength, \
mixing_image_prompt_and_vary_upscale, mixing_image_prompt_and_inpaint, \
debugging_cn_preprocessor, skipping_cn_preprocessor, controlnet_softness, canny_low_threshold, canny_high_threshold, \
refiner_swap_method, \
freeu_enabled, freeu_b1, freeu_b2, freeu_s1, freeu_s2, \
debugging_inpaint_preprocessor, inpaint_disable_initial_latent, inpaint_engine, inpaint_strength, inpaint_respective_field, \
inpaint_mask_upload_checkbox, invert_mask_checkbox, inpaint_erode_or_dilate
disable_preview, adm_scaler_positive, adm_scaler_negative, adm_scaler_end, adaptive_cfg, sampler_name, \
scheduler_name, generate_image_grid, overwrite_step, overwrite_switch, overwrite_width, overwrite_height, \
overwrite_vary_strength, overwrite_upscale_strength, \
mixing_image_prompt_and_vary_upscale, mixing_image_prompt_and_inpaint, \
debugging_cn_preprocessor, skipping_cn_preprocessor, controlnet_softness, canny_low_threshold, canny_high_threshold, \
refiner_swap_method, \
freeu_enabled, freeu_b1, freeu_b2, freeu_s1, freeu_s2, \
debugging_inpaint_preprocessor, inpaint_disable_initial_latent, inpaint_engine, inpaint_strength, inpaint_respective_field, \
inpaint_mask_upload_checkbox, invert_mask_checkbox, inpaint_erode_or_dilate = args
return

498
modules/async_worker.py
View File

@ -1,4 +1,8 @@
import threading
import re
from modules.patch import PatchSettings, patch_settings, patch_all
patch_all()
class AsyncTask:
@ -6,6 +10,8 @@ class AsyncTask:
self.args = args
self.yields = []
self.results = []
self.last_stop = False
self.processing = False
async_tasks = []
@ -14,9 +20,11 @@ async_tasks = []
def worker():
global async_tasks
import os
import traceback
import math
import numpy as np
import cv2
import torch
import time
import shared
@ -31,17 +39,24 @@ def worker():
import extras.preprocessors as preprocessors
import modules.inpaint_worker as inpaint_worker
import modules.constants as constants
import modules.advanced_parameters as advanced_parameters
import extras.ip_adapter as ip_adapter
import extras.face_crop
import fooocus_version
import args_manager
from modules.sdxl_styles import apply_style, apply_wildcards, fooocus_expansion
from extras.censor import default_censor
from modules.sdxl_styles import apply_style, get_random_style, fooocus_expansion, apply_arrays, random_style_name
from modules.private_logger import log
from extras.expansion import safe_str
from modules.util import remove_empty_str, HWC3, resize_image, \
get_image_shape_ceil, set_image_shape_ceil, get_shape_ceil, resample_image, erode_or_dilate
from modules.util import (remove_empty_str, HWC3, resize_image, get_image_shape_ceil, set_image_shape_ceil,
get_shape_ceil, resample_image, erode_or_dilate, get_enabled_loras,
parse_lora_references_from_prompt, apply_wildcards)
from modules.upscaler import perform_upscale
from modules.flags import Performance
from modules.meta_parser import get_metadata_parser, MetadataScheme
pid = os.getpid()
print(f'Started worker with PID {pid}')
try:
async_gradio_app = shared.gradio_root
@ -56,10 +71,15 @@ def worker():
print(f'[Fooocus] {text}')
async_task.yields.append(['preview', (number, text, None)])
def yield_result(async_task, imgs, do_not_show_finished_images=False):
def yield_result(async_task, imgs, black_out_nsfw, censor=True, do_not_show_finished_images=False,
progressbar_index=flags.preparation_step_count):
if not isinstance(imgs, list):
imgs = [imgs]
if censor and (modules.config.default_black_out_nsfw or black_out_nsfw):
progressbar(async_task, progressbar_index, 'Checking for NSFW content ...')
imgs = default_censor(imgs)
async_task.results = async_task.results + imgs
if do_not_show_finished_images:
@ -69,19 +89,20 @@ def worker():
return
def build_image_wall(async_task):
if not advanced_parameters.generate_image_grid:
results = []
if len(async_task.results) < 2:
return
results = async_task.results
if len(results) < 2:
return
for img in results:
for img in async_task.results:
if isinstance(img, str) and os.path.exists(img):
img = cv2.imread(img)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
if not isinstance(img, np.ndarray):
return
if img.ndim != 3:
return
results.append(img)
H, W, C = results[0].shape
@ -115,6 +136,7 @@ def worker():
@torch.inference_mode()
def handler(async_task):
execution_start_time = time.perf_counter()
async_task.processing = True
args = async_task.args
args.reverse()
@ -122,16 +144,19 @@ def worker():
prompt = args.pop()
negative_prompt = args.pop()
style_selections = args.pop()
performance_selection = args.pop()
performance_selection = Performance(args.pop())
aspect_ratios_selection = args.pop()
image_number = args.pop()
output_format = args.pop()
image_seed = args.pop()
read_wildcards_in_order = args.pop()
sharpness = args.pop()
guidance_scale = args.pop()
base_model_name = args.pop()
refiner_model_name = args.pop()
refiner_switch = args.pop()
loras = [[str(args.pop()), float(args.pop())] for _ in range(5)]
loras = get_enabled_loras([(bool(args.pop()), str(args.pop()), float(args.pop())) for _ in
range(modules.config.default_max_lora_number)])
input_image_checkbox = args.pop()
current_tab = args.pop()
uov_method = args.pop()
@ -141,8 +166,52 @@ def worker():
inpaint_additional_prompt = args.pop()
inpaint_mask_image_upload = args.pop()
disable_preview = args.pop()
disable_intermediate_results = args.pop()
disable_seed_increment = args.pop()
black_out_nsfw = args.pop()
adm_scaler_positive = args.pop()
adm_scaler_negative = args.pop()
adm_scaler_end = args.pop()
adaptive_cfg = args.pop()
clip_skip = args.pop()
sampler_name = args.pop()
scheduler_name = args.pop()
vae_name = args.pop()
overwrite_step = args.pop()
overwrite_switch = args.pop()
overwrite_width = args.pop()
overwrite_height = args.pop()
overwrite_vary_strength = args.pop()
overwrite_upscale_strength = args.pop()
mixing_image_prompt_and_vary_upscale = args.pop()
mixing_image_prompt_and_inpaint = args.pop()
debugging_cn_preprocessor = args.pop()
skipping_cn_preprocessor = args.pop()
canny_low_threshold = args.pop()
canny_high_threshold = args.pop()
refiner_swap_method = args.pop()
controlnet_softness = args.pop()
freeu_enabled = args.pop()
freeu_b1 = args.pop()
freeu_b2 = args.pop()
freeu_s1 = args.pop()
freeu_s2 = args.pop()
debugging_inpaint_preprocessor = args.pop()
inpaint_disable_initial_latent = args.pop()
inpaint_engine = args.pop()
inpaint_strength = args.pop()
inpaint_respective_field = args.pop()
inpaint_mask_upload_checkbox = args.pop()
invert_mask_checkbox = args.pop()
inpaint_erode_or_dilate = args.pop()
save_metadata_to_images = args.pop() if not args_manager.args.disable_metadata else False
metadata_scheme = MetadataScheme(
args.pop()) if not args_manager.args.disable_metadata else MetadataScheme.FOOOCUS
cn_tasks = {x: [] for x in flags.ip_list}
for _ in range(4):
for _ in range(flags.controlnet_image_count):
cn_img = args.pop()
cn_stop = args.pop()
cn_weight = args.pop()
@ -167,49 +236,84 @@ def worker():
print(f'Refiner disabled because base model and refiner are same.')
refiner_model_name = 'None'
assert performance_selection in ['Speed', 'Quality', 'Extreme Speed']
steps = performance_selection.steps()
steps = 30
performance_loras = []
if performance_selection == 'Speed':
steps = 30
if performance_selection == 'Quality':
steps = 60
if performance_selection == 'Extreme Speed':
if performance_selection == Performance.EXTREME_SPEED:
print('Enter LCM mode.')
progressbar(async_task, 1, 'Downloading LCM components ...')
loras += [(modules.config.downloading_sdxl_lcm_lora(), 1.0)]
performance_loras += [(modules.config.downloading_sdxl_lcm_lora(), 1.0)]
if refiner_model_name != 'None':
print(f'Refiner disabled in LCM mode.')
refiner_model_name = 'None'
sampler_name = advanced_parameters.sampler_name = 'lcm'
scheduler_name = advanced_parameters.scheduler_name = 'lcm'
modules.patch.sharpness = sharpness = 0.0
cfg_scale = guidance_scale = 1.0
modules.patch.adaptive_cfg = advanced_parameters.adaptive_cfg = 1.0
sampler_name = 'lcm'
scheduler_name = 'lcm'
sharpness = 0.0
guidance_scale = 1.0
adaptive_cfg = 1.0
refiner_switch = 1.0
modules.patch.positive_adm_scale = advanced_parameters.adm_scaler_positive = 1.0
modules.patch.negative_adm_scale = advanced_parameters.adm_scaler_negative = 1.0
modules.patch.adm_scaler_end = advanced_parameters.adm_scaler_end = 0.0
steps = 8
adm_scaler_positive = 1.0
adm_scaler_negative = 1.0
adm_scaler_end = 0.0
modules.patch.adaptive_cfg = advanced_parameters.adaptive_cfg
print(f'[Parameters] Adaptive CFG = {modules.patch.adaptive_cfg}')
elif performance_selection == Performance.LIGHTNING:
print('Enter Lightning mode.')
progressbar(async_task, 1, 'Downloading Lightning components ...')
performance_loras += [(modules.config.downloading_sdxl_lightning_lora(), 1.0)]
modules.patch.sharpness = sharpness
print(f'[Parameters] Sharpness = {modules.patch.sharpness}')
if refiner_model_name != 'None':
print(f'Refiner disabled in Lightning mode.')
modules.patch.positive_adm_scale = advanced_parameters.adm_scaler_positive
modules.patch.negative_adm_scale = advanced_parameters.adm_scaler_negative
modules.patch.adm_scaler_end = advanced_parameters.adm_scaler_end
refiner_model_name = 'None'
sampler_name = 'euler'
scheduler_name = 'sgm_uniform'
sharpness = 0.0
guidance_scale = 1.0
adaptive_cfg = 1.0
refiner_switch = 1.0
adm_scaler_positive = 1.0
adm_scaler_negative = 1.0
adm_scaler_end = 0.0
elif performance_selection == Performance.HYPER_SD:
print('Enter Hyper-SD mode.')
progressbar(async_task, 1, 'Downloading Hyper-SD components ...')
performance_loras += [(modules.config.downloading_sdxl_hyper_sd_lora(), 0.8)]
if refiner_model_name != 'None':
print(f'Refiner disabled in Hyper-SD mode.')
refiner_model_name = 'None'
sampler_name = 'dpmpp_sde_gpu'
scheduler_name = 'karras'
sharpness = 0.0
guidance_scale = 1.0
adaptive_cfg = 1.0
refiner_switch = 1.0
adm_scaler_positive = 1.0
adm_scaler_negative = 1.0
adm_scaler_end = 0.0
print(f'[Parameters] Adaptive CFG = {adaptive_cfg}')
print(f'[Parameters] CLIP Skip = {clip_skip}')
print(f'[Parameters] Sharpness = {sharpness}')
print(f'[Parameters] ControlNet Softness = {controlnet_softness}')
print(f'[Parameters] ADM Scale = '
f'{modules.patch.positive_adm_scale} : '
f'{modules.patch.negative_adm_scale} : '
f'{modules.patch.adm_scaler_end}')
f'{adm_scaler_positive} : '
f'{adm_scaler_negative} : '
f'{adm_scaler_end}')
patch_settings[pid] = PatchSettings(
sharpness,
adm_scaler_end,
adm_scaler_positive,
adm_scaler_negative,
controlnet_softness,
adaptive_cfg
)
cfg_scale = float(guidance_scale)
print(f'[Parameters] CFG = {cfg_scale}')
@ -222,10 +326,9 @@ def worker():
width, height = int(width), int(height)
skip_prompt_processing = False
refiner_swap_method = advanced_parameters.refiner_swap_method
inpaint_worker.current_task = None
inpaint_parameterized = advanced_parameters.inpaint_engine != 'None'
inpaint_parameterized = inpaint_engine != 'None'
inpaint_image = None
inpaint_mask = None
inpaint_head_model_path = None
@ -239,15 +342,12 @@ def worker():
seed = int(image_seed)
print(f'[Parameters] Seed = {seed}')
sampler_name = advanced_parameters.sampler_name
scheduler_name = advanced_parameters.scheduler_name
goals = []
tasks = []
if input_image_checkbox:
if (current_tab == 'uov' or (
current_tab == 'ip' and advanced_parameters.mixing_image_prompt_and_vary_upscale)) \
current_tab == 'ip' and mixing_image_prompt_and_vary_upscale)) \
and uov_method != flags.disabled and uov_input_image is not None:
uov_input_image = HWC3(uov_input_image)
if 'vary' in uov_method:
@ -257,26 +357,17 @@ def worker():
if 'fast' in uov_method:
skip_prompt_processing = True
else:
steps = 18
if performance_selection == 'Speed':
steps = 18
if performance_selection == 'Quality':
steps = 36
if performance_selection == 'Extreme Speed':
steps = 8
steps = performance_selection.steps_uov()
progressbar(async_task, 1, 'Downloading upscale models ...')
modules.config.downloading_upscale_model()
if (current_tab == 'inpaint' or (
current_tab == 'ip' and advanced_parameters.mixing_image_prompt_and_inpaint)) \
current_tab == 'ip' and mixing_image_prompt_and_inpaint)) \
and isinstance(inpaint_input_image, dict):
inpaint_image = inpaint_input_image['image']
inpaint_mask = inpaint_input_image['mask'][:, :, 0]
if advanced_parameters.inpaint_mask_upload_checkbox:
if inpaint_mask_upload_checkbox:
if isinstance(inpaint_mask_image_upload, np.ndarray):
if inpaint_mask_image_upload.ndim == 3:
H, W, C = inpaint_image.shape
@ -285,10 +376,10 @@ def worker():
inpaint_mask_image_upload = (inpaint_mask_image_upload > 127).astype(np.uint8) * 255
inpaint_mask = np.maximum(inpaint_mask, inpaint_mask_image_upload)
if int(advanced_parameters.inpaint_erode_or_dilate) != 0:
inpaint_mask = erode_or_dilate(inpaint_mask, advanced_parameters.inpaint_erode_or_dilate)
if int(inpaint_erode_or_dilate) != 0:
inpaint_mask = erode_or_dilate(inpaint_mask, inpaint_erode_or_dilate)
if advanced_parameters.invert_mask_checkbox:
if invert_mask_checkbox:
inpaint_mask = 255 - inpaint_mask
inpaint_image = HWC3(inpaint_image)
@ -299,12 +390,12 @@ def worker():
if inpaint_parameterized:
progressbar(async_task, 1, 'Downloading inpainter ...')
inpaint_head_model_path, inpaint_patch_model_path = modules.config.downloading_inpaint_models(
advanced_parameters.inpaint_engine)
inpaint_engine)
base_model_additional_loras += [(inpaint_patch_model_path, 1.0)]
print(f'[Inpaint] Current inpaint model is {inpaint_patch_model_path}')
if refiner_model_name == 'None':
use_synthetic_refiner = True
refiner_switch = 0.5
refiner_switch = 0.8
else:
inpaint_head_model_path, inpaint_patch_model_path = None, None
print(f'[Inpaint] Parameterized inpaint is disabled.')
@ -315,8 +406,8 @@ def worker():
prompt = inpaint_additional_prompt + '\n' + prompt
goals.append('inpaint')
if current_tab == 'ip' or \
advanced_parameters.mixing_image_prompt_and_inpaint or \
advanced_parameters.mixing_image_prompt_and_vary_upscale:
mixing_image_prompt_and_vary_upscale or \
mixing_image_prompt_and_inpaint:
goals.append('cn')
progressbar(async_task, 1, 'Downloading control models ...')
if len(cn_tasks[flags.cn_canny]) > 0:
@ -335,19 +426,19 @@ def worker():
ip_adapter.load_ip_adapter(clip_vision_path, ip_negative_path, ip_adapter_path)
ip_adapter.load_ip_adapter(clip_vision_path, ip_negative_path, ip_adapter_face_path)
if overwrite_step > 0:
steps = overwrite_step
switch = int(round(steps * refiner_switch))
if advanced_parameters.overwrite_step > 0:
steps = advanced_parameters.overwrite_step
if overwrite_switch > 0:
switch = overwrite_switch
if advanced_parameters.overwrite_switch > 0:
switch = advanced_parameters.overwrite_switch
if overwrite_width > 0:
width = overwrite_width
if advanced_parameters.overwrite_width > 0:
width = advanced_parameters.overwrite_width
if advanced_parameters.overwrite_height > 0:
height = advanced_parameters.overwrite_height
if overwrite_height > 0:
height = overwrite_height
print(f'[Parameters] Sampler = {sampler_name} - {scheduler_name}')
print(f'[Parameters] Steps = {steps} - {switch}')
@ -369,27 +460,43 @@ def worker():
extra_positive_prompts = prompts[1:] if len(prompts) > 1 else []
extra_negative_prompts = negative_prompts[1:] if len(negative_prompts) > 1 else []
progressbar(async_task, 3, 'Loading models ...')
progressbar(async_task, 2, 'Loading models ...')
loras, prompt = parse_lora_references_from_prompt(prompt, loras, modules.config.default_max_lora_number)
loras += performance_loras
pipeline.refresh_everything(refiner_model_name=refiner_model_name, base_model_name=base_model_name,
loras=loras, base_model_additional_loras=base_model_additional_loras,
use_synthetic_refiner=use_synthetic_refiner)
use_synthetic_refiner=use_synthetic_refiner, vae_name=vae_name)
pipeline.set_clip_skip(clip_skip)
progressbar(async_task, 3, 'Processing prompts ...')
tasks = []
for i in range(image_number):
task_seed = (seed + i) % (constants.MAX_SEED + 1) # randint is inclusive, % is not
task_rng = random.Random(task_seed) # may bind to inpaint noise in the future
task_prompt = apply_wildcards(prompt, task_rng)
task_negative_prompt = apply_wildcards(negative_prompt, task_rng)
task_extra_positive_prompts = [apply_wildcards(pmt, task_rng) for pmt in extra_positive_prompts]
task_extra_negative_prompts = [apply_wildcards(pmt, task_rng) for pmt in extra_negative_prompts]
for i in range(image_number):
if disable_seed_increment:
task_seed = seed % (constants.MAX_SEED + 1)
else:
task_seed = (seed + i) % (constants.MAX_SEED + 1) # randint is inclusive, % is not
task_rng = random.Random(task_seed) # may bind to inpaint noise in the future
task_prompt = apply_wildcards(prompt, task_rng, i, read_wildcards_in_order)
task_prompt = apply_arrays(task_prompt, i)
task_negative_prompt = apply_wildcards(negative_prompt, task_rng, i, read_wildcards_in_order)
task_extra_positive_prompts = [apply_wildcards(pmt, task_rng, i, read_wildcards_in_order) for pmt in
extra_positive_prompts]
task_extra_negative_prompts = [apply_wildcards(pmt, task_rng, i, read_wildcards_in_order) for pmt in
extra_negative_prompts]
positive_basic_workloads = []
negative_basic_workloads = []
task_styles = style_selections.copy()
if use_style:
for s in style_selections:
for i, s in enumerate(task_styles):
if s == random_style_name:
s = get_random_style(task_rng)
task_styles[i] = s
p, n = apply_style(s, positive=task_prompt)
positive_basic_workloads = positive_basic_workloads + p
negative_basic_workloads = negative_basic_workloads + n
@ -417,37 +524,38 @@ def worker():
negative_top_k=len(negative_basic_workloads),
log_positive_prompt='\n'.join([task_prompt] + task_extra_positive_prompts),
log_negative_prompt='\n'.join([task_negative_prompt] + task_extra_negative_prompts),
styles=task_styles
))
if use_expansion:
for i, t in enumerate(tasks):
progressbar(async_task, 5, f'Preparing Fooocus text #{i + 1} ...')
progressbar(async_task, 4, f'Preparing Fooocus text #{i + 1} ...')
expansion = pipeline.final_expansion(t['task_prompt'], t['task_seed'])
print(f'[Prompt Expansion] {expansion}')
t['expansion'] = expansion
t['positive'] = copy.deepcopy(t['positive']) + [expansion] # Deep copy.
for i, t in enumerate(tasks):
progressbar(async_task, 7, f'Encoding positive #{i + 1} ...')
progressbar(async_task, 5, f'Encoding positive #{i + 1} ...')
t['c'] = pipeline.clip_encode(texts=t['positive'], pool_top_k=t['positive_top_k'])
for i, t in enumerate(tasks):
if abs(float(cfg_scale) - 1.0) < 1e-4:
t['uc'] = pipeline.clone_cond(t['c'])
else:
progressbar(async_task, 10, f'Encoding negative #{i + 1} ...')
progressbar(async_task, 6, f'Encoding negative #{i + 1} ...')
t['uc'] = pipeline.clip_encode(texts=t['negative'], pool_top_k=t['negative_top_k'])
if len(goals) > 0:
progressbar(async_task, 13, 'Image processing ...')
progressbar(async_task, 7, 'Image processing ...')
if 'vary' in goals:
if 'subtle' in uov_method:
denoising_strength = 0.5
if 'strong' in uov_method:
denoising_strength = 0.85
if advanced_parameters.overwrite_vary_strength > 0:
denoising_strength = advanced_parameters.overwrite_vary_strength
if overwrite_vary_strength > 0:
denoising_strength = overwrite_vary_strength
shape_ceil = get_image_shape_ceil(uov_input_image)
if shape_ceil < 1024:
@ -460,7 +568,7 @@ def worker():
uov_input_image = set_image_shape_ceil(uov_input_image, shape_ceil)
initial_pixels = core.numpy_to_pytorch(uov_input_image)
progressbar(async_task, 13, 'VAE encoding ...')
progressbar(async_task, 8, 'VAE encoding ...')
candidate_vae, _ = pipeline.get_candidate_vae(
steps=steps,
@ -477,7 +585,7 @@ def worker():
if 'upscale' in goals:
H, W, C = uov_input_image.shape
progressbar(async_task, 13, f'Upscaling image from {str((H, W))} ...')
progressbar(async_task, 9, f'Upscaling image from {str((H, W))} ...')
uov_input_image = perform_upscale(uov_input_image)
print(f'Image upscaled.')
@ -510,19 +618,23 @@ def worker():
direct_return = False
if direct_return:
d = [('Upscale (Fast)', '2x')]
log(uov_input_image, d)
yield_result(async_task, uov_input_image, do_not_show_finished_images=True)
d = [('Upscale (Fast)', 'upscale_fast', '2x')]
if modules.config.default_black_out_nsfw or black_out_nsfw:
progressbar(async_task, 100, 'Checking for NSFW content ...')
uov_input_image = default_censor(uov_input_image)
progressbar(async_task, 100, 'Saving image to system ...')
uov_input_image_path = log(uov_input_image, d, output_format=output_format)
yield_result(async_task, uov_input_image_path, black_out_nsfw, False, do_not_show_finished_images=True)
return
tiled = True
denoising_strength = 0.382
if advanced_parameters.overwrite_upscale_strength > 0:
denoising_strength = advanced_parameters.overwrite_upscale_strength
if overwrite_upscale_strength > 0:
denoising_strength = overwrite_upscale_strength
initial_pixels = core.numpy_to_pytorch(uov_input_image)
progressbar(async_task, 13, 'VAE encoding ...')
progressbar(async_task, 10, 'VAE encoding ...')
candidate_vae, _ = pipeline.get_candidate_vae(
steps=steps,
@ -553,34 +665,34 @@ def worker():
H, W, C = inpaint_image.shape
if 'left' in outpaint_selections:
inpaint_image = np.pad(inpaint_image, [[0, 0], [int(H * 0.3), 0], [0, 0]], mode='edge')
inpaint_mask = np.pad(inpaint_mask, [[0, 0], [int(H * 0.3), 0]], mode='constant',
inpaint_image = np.pad(inpaint_image, [[0, 0], [int(W * 0.3), 0], [0, 0]], mode='edge')
inpaint_mask = np.pad(inpaint_mask, [[0, 0], [int(W * 0.3), 0]], mode='constant',
constant_values=255)
if 'right' in outpaint_selections:
inpaint_image = np.pad(inpaint_image, [[0, 0], [0, int(H * 0.3)], [0, 0]], mode='edge')
inpaint_mask = np.pad(inpaint_mask, [[0, 0], [0, int(H * 0.3)]], mode='constant',
inpaint_image = np.pad(inpaint_image, [[0, 0], [0, int(W * 0.3)], [0, 0]], mode='edge')
inpaint_mask = np.pad(inpaint_mask, [[0, 0], [0, int(W * 0.3)]], mode='constant',
constant_values=255)
inpaint_image = np.ascontiguousarray(inpaint_image.copy())
inpaint_mask = np.ascontiguousarray(inpaint_mask.copy())
advanced_parameters.inpaint_strength = 1.0
advanced_parameters.inpaint_respective_field = 1.0
inpaint_strength = 1.0
inpaint_respective_field = 1.0
denoising_strength = advanced_parameters.inpaint_strength
denoising_strength = inpaint_strength
inpaint_worker.current_task = inpaint_worker.InpaintWorker(
image=inpaint_image,
mask=inpaint_mask,
use_fill=denoising_strength > 0.99,
k=advanced_parameters.inpaint_respective_field
k=inpaint_respective_field
)
if advanced_parameters.debugging_inpaint_preprocessor:
yield_result(async_task, inpaint_worker.current_task.visualize_mask_processing(),
if debugging_inpaint_preprocessor:
yield_result(async_task, inpaint_worker.current_task.visualize_mask_processing(), black_out_nsfw,
do_not_show_finished_images=True)
return
progressbar(async_task, 13, 'VAE Inpaint encoding ...')
progressbar(async_task, 11, 'VAE Inpaint encoding ...')
inpaint_pixel_fill = core.numpy_to_pytorch(inpaint_worker.current_task.interested_fill)
inpaint_pixel_image = core.numpy_to_pytorch(inpaint_worker.current_task.interested_image)
@ -600,7 +712,7 @@ def worker():
latent_swap = None
if candidate_vae_swap is not None:
progressbar(async_task, 13, 'VAE SD15 encoding ...')
progressbar(async_task, 12, 'VAE SD15 encoding ...')
latent_swap = core.encode_vae(
vae=candidate_vae_swap,
pixels=inpaint_pixel_fill)['samples']
@ -621,7 +733,7 @@ def worker():
model=pipeline.final_unet
)
if not advanced_parameters.inpaint_disable_initial_latent:
if not inpaint_disable_initial_latent:
initial_latent = {'samples': latent_fill}
B, C, H, W = latent_fill.shape
@ -634,25 +746,25 @@ def worker():
cn_img, cn_stop, cn_weight = task
cn_img = resize_image(HWC3(cn_img), width=width, height=height)
if not advanced_parameters.skipping_cn_preprocessor:
cn_img = preprocessors.canny_pyramid(cn_img)
if not skipping_cn_preprocessor:
cn_img = preprocessors.canny_pyramid(cn_img, canny_low_threshold, canny_high_threshold)
cn_img = HWC3(cn_img)
task[0] = core.numpy_to_pytorch(cn_img)
if advanced_parameters.debugging_cn_preprocessor:
yield_result(async_task, cn_img, do_not_show_finished_images=True)
if debugging_cn_preprocessor:
yield_result(async_task, cn_img, black_out_nsfw, do_not_show_finished_images=True)
return
for task in cn_tasks[flags.cn_cpds]:
cn_img, cn_stop, cn_weight = task
cn_img = resize_image(HWC3(cn_img), width=width, height=height)
if not advanced_parameters.skipping_cn_preprocessor:
if not skipping_cn_preprocessor:
cn_img = preprocessors.cpds(cn_img)
cn_img = HWC3(cn_img)
task[0] = core.numpy_to_pytorch(cn_img)
if advanced_parameters.debugging_cn_preprocessor:
yield_result(async_task, cn_img, do_not_show_finished_images=True)
if debugging_cn_preprocessor:
yield_result(async_task, cn_img, black_out_nsfw, do_not_show_finished_images=True)
return
for task in cn_tasks[flags.cn_ip]:
cn_img, cn_stop, cn_weight = task
@ -662,22 +774,22 @@ def worker():
cn_img = resize_image(cn_img, width=224, height=224, resize_mode=0)
task[0] = ip_adapter.preprocess(cn_img, ip_adapter_path=ip_adapter_path)
if advanced_parameters.debugging_cn_preprocessor:
yield_result(async_task, cn_img, do_not_show_finished_images=True)
if debugging_cn_preprocessor:
yield_result(async_task, cn_img, black_out_nsfw, do_not_show_finished_images=True)
return
for task in cn_tasks[flags.cn_ip_face]:
cn_img, cn_stop, cn_weight = task
cn_img = HWC3(cn_img)
if not advanced_parameters.skipping_cn_preprocessor:
if not skipping_cn_preprocessor:
cn_img = extras.face_crop.crop_image(cn_img)
# https://github.com/tencent-ailab/IP-Adapter/blob/d580c50a291566bbf9fc7ac0f760506607297e6d/README.md?plain=1#L75
cn_img = resize_image(cn_img, width=224, height=224, resize_mode=0)
task[0] = ip_adapter.preprocess(cn_img, ip_adapter_path=ip_adapter_face_path)
if advanced_parameters.debugging_cn_preprocessor:
yield_result(async_task, cn_img, do_not_show_finished_images=True)
if debugging_cn_preprocessor:
yield_result(async_task, cn_img, black_out_nsfw, do_not_show_finished_images=True)
return
all_ip_tasks = cn_tasks[flags.cn_ip] + cn_tasks[flags.cn_ip_face]
@ -685,14 +797,14 @@ def worker():
if len(all_ip_tasks) > 0:
pipeline.final_unet = ip_adapter.patch_model(pipeline.final_unet, all_ip_tasks)
if advanced_parameters.freeu_enabled:
if freeu_enabled:
print(f'FreeU is enabled!')
pipeline.final_unet = core.apply_freeu(
pipeline.final_unet,
advanced_parameters.freeu_b1,
advanced_parameters.freeu_b2,
advanced_parameters.freeu_s1,
advanced_parameters.freeu_s2
freeu_b1,
freeu_b2,
freeu_s1,
freeu_s2
)
all_steps = steps * image_number
@ -712,33 +824,36 @@ def worker():
final_sampler_name = sampler_name
final_scheduler_name = scheduler_name
if scheduler_name == 'lcm':
if scheduler_name in ['lcm', 'tcd']:
final_scheduler_name = 'sgm_uniform'
if pipeline.final_unet is not None:
pipeline.final_unet = core.opModelSamplingDiscrete.patch(
pipeline.final_unet,
sampling='lcm',
sampling=scheduler_name,
zsnr=False)[0]
if pipeline.final_refiner_unet is not None:
pipeline.final_refiner_unet = core.opModelSamplingDiscrete.patch(
pipeline.final_refiner_unet,
sampling='lcm',
sampling=scheduler_name,
zsnr=False)[0]
print('Using lcm scheduler.')
print(f'Using {scheduler_name} scheduler.')
async_task.yields.append(['preview', (13, 'Moving model to GPU ...', None)])
async_task.yields.append(['preview', (flags.preparation_step_count, 'Moving model to GPU ...', None)])
def callback(step, x0, x, total_steps, y):
done_steps = current_task_id * steps + step
async_task.yields.append(['preview', (
int(15.0 + 85.0 * float(done_steps) / float(all_steps)),
f'Step {step}/{total_steps} in the {current_task_id + 1}-th Sampling',
y)])
int(flags.preparation_step_count + (100 - flags.preparation_step_count) * float(done_steps) / float(all_steps)),
f'Sampling step {step + 1}/{total_steps}, image {current_task_id + 1}/{image_number} ...', y)])
for current_task_id, task in enumerate(tasks):
current_progress = int(flags.preparation_step_count + (100 - flags.preparation_step_count) * float(current_task_id * steps) / float(all_steps))
progressbar(async_task, current_progress, f'Preparing task {current_task_id + 1}/{image_number} ...')
execution_start_time = time.perf_counter()
try:
if async_task.last_stop is not False:
ldm_patched.modules.model_management.interrupt_current_processing()
positive_cond, negative_cond = task['c'], task['uc']
if 'cn' in goals:
@ -766,7 +881,8 @@ def worker():
denoise=denoising_strength,
tiled=tiled,
cfg_scale=cfg_scale,
refiner_swap_method=refiner_swap_method
refiner_swap_method=refiner_swap_method,
disable_preview=disable_preview
)
del task['c'], task['uc'], positive_cond, negative_cond # Save memory
@ -774,37 +890,75 @@ def worker():
if inpaint_worker.current_task is not None:
imgs = [inpaint_worker.current_task.post_process(x) for x in imgs]
img_paths = []
current_progress = int(flags.preparation_step_count + (100 - flags.preparation_step_count) * float((current_task_id + 1) * steps) / float(all_steps))
if modules.config.default_black_out_nsfw or black_out_nsfw:
progressbar(async_task, current_progress, 'Checking for NSFW content ...')
imgs = default_censor(imgs)
progressbar(async_task, current_progress, f'Saving image {current_task_id + 1}/{image_number} to system ...')
for x in imgs:
d = [
('Prompt', task['log_positive_prompt']),
('Negative Prompt', task['log_negative_prompt']),
('Fooocus V2 Expansion', task['expansion']),
('Styles', str(raw_style_selections)),
('Performance', performance_selection),
('Resolution', str((width, height))),
('Sharpness', sharpness),
('Guidance Scale', guidance_scale),
('ADM Guidance', str((
modules.patch.positive_adm_scale,
modules.patch.negative_adm_scale,
modules.patch.adm_scaler_end))),
('Base Model', base_model_name),
('Refiner Model', refiner_model_name),
('Refiner Switch', refiner_switch),
('Sampler', sampler_name),
('Scheduler', scheduler_name),
('Seed', task['task_seed']),
]
d = [('Prompt', 'prompt', task['log_positive_prompt']),
('Negative Prompt', 'negative_prompt', task['log_negative_prompt']),
('Fooocus V2 Expansion', 'prompt_expansion', task['expansion']),
('Styles', 'styles',
str(task['styles'] if not use_expansion else [fooocus_expansion] + task['styles'])),
('Performance', 'performance', performance_selection.value)]
if performance_selection.steps() != steps:
d.append(('Steps', 'steps', steps))
d += [('Resolution', 'resolution', str((width, height))),
('Guidance Scale', 'guidance_scale', guidance_scale),
('Sharpness', 'sharpness', sharpness),
('ADM Guidance', 'adm_guidance', str((
modules.patch.patch_settings[pid].positive_adm_scale,
modules.patch.patch_settings[pid].negative_adm_scale,
modules.patch.patch_settings[pid].adm_scaler_end))),
('Base Model', 'base_model', base_model_name),
('Refiner Model', 'refiner_model', refiner_model_name),
('Refiner Switch', 'refiner_switch', refiner_switch)]
if refiner_model_name != 'None':
if overwrite_switch > 0:
d.append(('Overwrite Switch', 'overwrite_switch', overwrite_switch))
if refiner_swap_method != flags.refiner_swap_method:
d.append(('Refiner Swap Method', 'refiner_swap_method', refiner_swap_method))
if modules.patch.patch_settings[pid].adaptive_cfg != modules.config.default_cfg_tsnr:
d.append(
('CFG Mimicking from TSNR', 'adaptive_cfg', modules.patch.patch_settings[pid].adaptive_cfg))
if clip_skip > 1:
d.append(('CLIP Skip', 'clip_skip', clip_skip))
d.append(('Sampler', 'sampler', sampler_name))
d.append(('Scheduler', 'scheduler', scheduler_name))
d.append(('VAE', 'vae', vae_name))
d.append(('Seed', 'seed', str(task['task_seed'])))
if freeu_enabled:
d.append(('FreeU', 'freeu', str((freeu_b1, freeu_b2, freeu_s1, freeu_s2))))
for li, (n, w) in enumerate(loras):
if n != 'None':
d.append((f'LoRA {li + 1}', f'{n} : {w}'))
d.append(('Version', 'v' + fooocus_version.version))
log(x, d)
d.append((f'LoRA {li + 1}', f'lora_combined_{li + 1}', f'{n} : {w}'))
yield_result(async_task, imgs, do_not_show_finished_images=len(tasks) == 1)
metadata_parser = None
if save_metadata_to_images:
metadata_parser = modules.meta_parser.get_metadata_parser(metadata_scheme)
metadata_parser.set_data(task['log_positive_prompt'], task['positive'],
task['log_negative_prompt'], task['negative'],
steps, base_model_name, refiner_model_name, loras, vae_name)
d.append(('Metadata Scheme', 'metadata_scheme',
metadata_scheme.value if save_metadata_to_images else save_metadata_to_images))
d.append(('Version', 'version', 'Fooocus v' + fooocus_version.version))
img_paths.append(log(x, d, metadata_parser, output_format, task))
yield_result(async_task, img_paths, black_out_nsfw, False,
do_not_show_finished_images=len(tasks) == 1 or disable_intermediate_results)
except ldm_patched.modules.model_management.InterruptProcessingException as e:
if shared.last_stop == 'skip':
if async_task.last_stop == 'skip':
print('User skipped')
async_task.last_stop = False
continue
else:
print('User stopped')
@ -812,21 +966,27 @@ def worker():
execution_time = time.perf_counter() - execution_start_time
print(f'Generating and saving time: {execution_time:.2f} seconds')
async_task.processing = False
return
while True:
time.sleep(0.01)
if len(async_tasks) > 0:
task = async_tasks.pop(0)
generate_image_grid = task.args.pop(0)
try:
handler(task)
build_image_wall(task)
if generate_image_grid:
build_image_wall(task)
task.yields.append(['finish', task.results])
pipeline.prepare_text_encoder(async_call=True)
except:
traceback.print_exc()
task.yields.append(['finish', task.results])
finally:
if pid in modules.patch.patch_settings:
del modules.patch.patch_settings[pid]
pass

358
modules/config.py
View File

@ -3,15 +3,26 @@ import json
import math
import numbers
import args_manager
import tempfile
import modules.flags
import modules.sdxl_styles
from modules.model_loader import load_file_from_url
from modules.util import get_files_from_folder
from modules.extra_utils import makedirs_with_log, get_files_from_folder
from modules.flags import OutputFormat, Performance, MetadataScheme
config_path = os.path.abspath("./config.txt")
config_example_path = os.path.abspath("config_modification_tutorial.txt")
def get_config_path(key, default_value):
env = os.getenv(key)
if env is not None and isinstance(env, str):
print(f"Environment: {key} = {env}")
return env
else:
return os.path.abspath(default_value)
wildcards_max_bfs_depth = 64
config_path = get_config_path('config_path', "./config.txt")
config_example_path = get_config_path('config_example_path', "config_modification_tutorial.txt")
config_dict = {}
always_save_keys = []
visited_keys = []
@ -86,23 +97,50 @@ def try_load_deprecated_user_path_config():
try_load_deprecated_user_path_config()
def get_presets():
preset_folder = 'presets'
presets = ['initial']
if not os.path.exists(preset_folder):
print('No presets found.')
return presets
return presets + [f[:f.index('.json')] for f in os.listdir(preset_folder) if f.endswith('.json')]
def try_get_preset_content(preset):
if isinstance(preset, str):
preset_path = os.path.abspath(f'./presets/{preset}.json')
try:
if os.path.exists(preset_path):
with open(preset_path, "r", encoding="utf-8") as json_file:
json_content = json.load(json_file)
print(f'Loaded preset: {preset_path}')
return json_content
else:
raise FileNotFoundError
except Exception as e:
print(f'Load preset [{preset_path}] failed')
print(e)
return {}
available_presets = get_presets()
preset = args_manager.args.preset
config_dict.update(try_get_preset_content(preset))
if isinstance(preset, str):
preset_path = os.path.abspath(f'./presets/{preset}.json')
try:
if os.path.exists(preset_path):
with open(preset_path, "r", encoding="utf-8") as json_file:
config_dict.update(json.load(json_file))
print(f'Loaded preset: {preset_path}')
else:
raise FileNotFoundError
except Exception as e:
print(f'Load preset [{preset_path}] failed')
print(e)
def get_path_output() -> str:
"""
Checking output path argument and overriding default path.
"""
global config_dict
path_output = get_dir_or_set_default('path_outputs', '../outputs/', make_directory=True)
if args_manager.args.output_path:
print(f'Overriding config value path_outputs with {args_manager.args.output_path}')
config_dict['path_outputs'] = path_output = args_manager.args.output_path
return path_output
def get_dir_or_set_default(key, default_value):
def get_dir_or_set_default(key, default_value, as_array=False, make_directory=False):
global config_dict, visited_keys, always_save_keys
if key not in visited_keys:
@ -111,28 +149,55 @@ def get_dir_or_set_default(key, default_value):
if key not in always_save_keys:
always_save_keys.append(key)
v = config_dict.get(key, None)
if isinstance(v, str) and os.path.exists(v) and os.path.isdir(v):
return v
v = os.getenv(key)
if v is not None:
print(f"Environment: {key} = {v}")
config_dict[key] = v
else:
v = config_dict.get(key, None)
if isinstance(v, str):
if make_directory:
makedirs_with_log(v)
if os.path.exists(v) and os.path.isdir(v):
return v if not as_array else [v]
elif isinstance(v, list):
if make_directory:
for d in v:
makedirs_with_log(d)
if all([os.path.exists(d) and os.path.isdir(d) for d in v]):
return v
if v is not None:
print(f'Failed to load config key: {json.dumps({key:v})} is invalid or does not exist; will use {json.dumps({key:default_value})} instead.')
if isinstance(default_value, list):
dp = []
for path in default_value:
abs_path = os.path.abspath(os.path.join(os.path.dirname(__file__), path))
dp.append(abs_path)
os.makedirs(abs_path, exist_ok=True)
else:
if v is not None:
print(f'Failed to load config key: {json.dumps({key:v})} is invalid or does not exist; will use {json.dumps({key:default_value})} instead.')
dp = os.path.abspath(os.path.join(os.path.dirname(__file__), default_value))
os.makedirs(dp, exist_ok=True)
config_dict[key] = dp
return dp
if as_array:
dp = [dp]
config_dict[key] = dp
return dp
path_checkpoints = get_dir_or_set_default('path_checkpoints', '../models/checkpoints/')
path_loras = get_dir_or_set_default('path_loras', '../models/loras/')
paths_checkpoints = get_dir_or_set_default('path_checkpoints', ['../models/checkpoints/'], True)
paths_loras = get_dir_or_set_default('path_loras', ['../models/loras/'], True)
path_embeddings = get_dir_or_set_default('path_embeddings', '../models/embeddings/')
path_vae_approx = get_dir_or_set_default('path_vae_approx', '../models/vae_approx/')
path_vae = get_dir_or_set_default('path_vae', '../models/vae/')
path_upscale_models = get_dir_or_set_default('path_upscale_models', '../models/upscale_models/')
path_inpaint = get_dir_or_set_default('path_inpaint', '../models/inpaint/')
path_controlnet = get_dir_or_set_default('path_controlnet', '../models/controlnet/')
path_clip_vision = get_dir_or_set_default('path_clip_vision', '../models/clip_vision/')
path_fooocus_expansion = get_dir_or_set_default('path_fooocus_expansion', '../models/prompt_expansion/fooocus_expansion')
path_outputs = get_dir_or_set_default('path_outputs', '../outputs/')
path_wildcards = get_dir_or_set_default('path_wildcards', '../wildcards/')
path_safety_checker = get_dir_or_set_default('path_safety_checker', '../models/safety_checker/')
path_outputs = get_path_output()
def get_config_item_or_set_default(key, default_value, validator, disable_empty_as_none=False):
@ -141,6 +206,11 @@ def get_config_item_or_set_default(key, default_value, validator, disable_empty_
if key not in visited_keys:
visited_keys.append(key)
v = os.getenv(key)
if v is not None:
print(f"Environment: {key} = {v}")
config_dict[key] = v
if key not in config_dict:
config_dict[key] = default_value
return default_value
@ -158,7 +228,37 @@ def get_config_item_or_set_default(key, default_value, validator, disable_empty_
return default_value
default_base_model_name = get_config_item_or_set_default(
def init_temp_path(path: str | None, default_path: str) -> str:
if args_manager.args.temp_path:
path = args_manager.args.temp_path
if path != '' and path != default_path:
try:
if not os.path.isabs(path):
path = os.path.abspath(path)
os.makedirs(path, exist_ok=True)
print(f'Using temp path {path}')
return path
except Exception as e:
print(f'Could not create temp path {path}. Reason: {e}')
print(f'Using default temp path {default_path} instead.')
os.makedirs(default_path, exist_ok=True)
return default_path
default_temp_path = os.path.join(tempfile.gettempdir(), 'fooocus')
temp_path = init_temp_path(get_config_item_or_set_default(
key='temp_path',
default_value=default_temp_path,
validator=lambda x: isinstance(x, str),
), default_temp_path)
temp_path_cleanup_on_launch = get_config_item_or_set_default(
key='temp_path_cleanup_on_launch',
default_value=True,
validator=lambda x: isinstance(x, bool)
)
default_base_model_name = default_model = get_config_item_or_set_default(
key='default_model',
default_value='model.safetensors',
validator=lambda x: isinstance(x, str)
@ -168,7 +268,7 @@ previous_default_models = get_config_item_or_set_default(
default_value=[],
validator=lambda x: isinstance(x, list) and all(isinstance(k, str) for k in x)
)
default_refiner_model_name = get_config_item_or_set_default(
default_refiner_model_name = default_refiner = get_config_item_or_set_default(
key='default_refiner',
default_value='None',
validator=lambda x: isinstance(x, str)
@ -178,31 +278,55 @@ default_refiner_switch = get_config_item_or_set_default(
default_value=0.8,
validator=lambda x: isinstance(x, numbers.Number) and 0 <= x <= 1
)
default_loras_min_weight = get_config_item_or_set_default(
key='default_loras_min_weight',
default_value=-2,
validator=lambda x: isinstance(x, numbers.Number) and -10 <= x <= 10
)
default_loras_max_weight = get_config_item_or_set_default(
key='default_loras_max_weight',
default_value=2,
validator=lambda x: isinstance(x, numbers.Number) and -10 <= x <= 10
)
default_loras = get_config_item_or_set_default(
key='default_loras',
default_value=[
[
True,
"None",
1.0
],
[
True,
"None",
1.0
],
[
True,
"None",
1.0
],
[
True,
"None",
1.0
],
[
True,
"None",
1.0
]
],
validator=lambda x: isinstance(x, list) and all(len(y) == 2 and isinstance(y[0], str) and isinstance(y[1], numbers.Number) for y in x)
validator=lambda x: isinstance(x, list) and all(
len(y) == 3 and isinstance(y[0], bool) and isinstance(y[1], str) and isinstance(y[2], numbers.Number)
or len(y) == 2 and isinstance(y[0], str) and isinstance(y[1], numbers.Number)
for y in x)
)
default_loras = [(y[0], y[1], y[2]) if len(y) == 3 else (True, y[0], y[1]) for y in default_loras]
default_max_lora_number = get_config_item_or_set_default(
key='default_max_lora_number',
default_value=len(default_loras) if isinstance(default_loras, list) and len(default_loras) > 0 else 5,
validator=lambda x: isinstance(x, int) and x >= 1
)
default_cfg_scale = get_config_item_or_set_default(
key='default_cfg_scale',
@ -224,6 +348,11 @@ default_scheduler = get_config_item_or_set_default(
default_value='karras',
validator=lambda x: x in modules.flags.scheduler_list
)
default_vae = get_config_item_or_set_default(
key='default_vae',
default_value=modules.flags.default_vae,
validator=lambda x: isinstance(x, str)
)
default_styles = get_config_item_or_set_default(
key='default_styles',
default_value=[
@ -247,8 +376,8 @@ default_prompt = get_config_item_or_set_default(
)
default_performance = get_config_item_or_set_default(
key='default_performance',
default_value='Speed',
validator=lambda x: x in modules.flags.performance_selections
default_value=Performance.SPEED.value,
validator=lambda x: x in Performance.list()
)
default_advanced_checkbox = get_config_item_or_set_default(
key='default_advanced_checkbox',
@ -260,6 +389,11 @@ default_max_image_number = get_config_item_or_set_default(
default_value=32,
validator=lambda x: isinstance(x, int) and x >= 1
)
default_output_format = get_config_item_or_set_default(
key='default_output_format',
default_value='png',
validator=lambda x: x in OutputFormat.list()
)
default_image_number = get_config_item_or_set_default(
key='default_image_number',
default_value=2,
@ -282,13 +416,7 @@ embeddings_downloads = get_config_item_or_set_default(
)
available_aspect_ratios = get_config_item_or_set_default(
key='available_aspect_ratios',
default_value=[
'704*1408', '704*1344', '768*1344', '768*1280', '832*1216', '832*1152',
'896*1152', '896*1088', '960*1088', '960*1024', '1024*1024', '1024*960',
'1088*960', '1088*896', '1152*896', '1152*832', '1216*832', '1280*768',
'1344*768', '1344*704', '1408*704', '1472*704', '1536*640', '1600*640',
'1664*576', '1728*576'
],
default_value=modules.flags.sdxl_aspect_ratios,
validator=lambda x: isinstance(x, list) and all('*' in v for v in x) and len(x) > 1
)
default_aspect_ratio = get_config_item_or_set_default(
@ -306,6 +434,11 @@ default_cfg_tsnr = get_config_item_or_set_default(
default_value=7.0,
validator=lambda x: isinstance(x, numbers.Number)
)
default_clip_skip = get_config_item_or_set_default(
key='default_clip_skip',
default_value=2,
validator=lambda x: isinstance(x, int) and 1 <= x <= modules.flags.clip_skip_max
)
default_overwrite_step = get_config_item_or_set_default(
key='default_overwrite_step',
default_value=-1,
@ -323,30 +456,58 @@ example_inpaint_prompts = get_config_item_or_set_default(
],
validator=lambda x: isinstance(x, list) and all(isinstance(v, str) for v in x)
)
default_black_out_nsfw = get_config_item_or_set_default(
key='default_black_out_nsfw',
default_value=False,
validator=lambda x: isinstance(x, bool)
)
default_save_metadata_to_images = get_config_item_or_set_default(
key='default_save_metadata_to_images',
default_value=False,
validator=lambda x: isinstance(x, bool)
)
default_metadata_scheme = get_config_item_or_set_default(
key='default_metadata_scheme',
default_value=MetadataScheme.FOOOCUS.value,
validator=lambda x: x in [y[1] for y in modules.flags.metadata_scheme if y[1] == x]
)
metadata_created_by = get_config_item_or_set_default(
key='metadata_created_by',
default_value='',
validator=lambda x: isinstance(x, str)
)
example_inpaint_prompts = [[x] for x in example_inpaint_prompts]
config_dict["default_loras"] = default_loras = default_loras[:5] + [['None', 1.0] for _ in range(5 - len(default_loras))]
possible_preset_keys = [
"default_model",
"default_refiner",
"default_refiner_switch",
"default_loras",
"default_cfg_scale",
"default_sample_sharpness",
"default_sampler",
"default_scheduler",
"default_performance",
"default_prompt",
"default_prompt_negative",
"default_styles",
"default_aspect_ratio",
"checkpoint_downloads",
"embeddings_downloads",
"lora_downloads",
]
config_dict["default_loras"] = default_loras = default_loras[:default_max_lora_number] + [[True, 'None', 1.0] for _ in range(default_max_lora_number - len(default_loras))]
# mapping config to meta parameter
possible_preset_keys = {
"default_model": "base_model",
"default_refiner": "refiner_model",
"default_refiner_switch": "refiner_switch",
"previous_default_models": "previous_default_models",
"default_loras_min_weight": "default_loras_min_weight",
"default_loras_max_weight": "default_loras_max_weight",
"default_loras": "<processed>",
"default_cfg_scale": "guidance_scale",
"default_sample_sharpness": "sharpness",
"default_cfg_tsnr": "adaptive_cfg",
"default_clip_skip": "clip_skip",
"default_sampler": "sampler",
"default_scheduler": "scheduler",
"default_overwrite_step": "steps",
"default_performance": "performance",
"default_image_number": "image_number",
"default_prompt": "prompt",
"default_prompt_negative": "negative_prompt",
"default_styles": "styles",
"default_aspect_ratio": "resolution",
"default_save_metadata_to_images": "default_save_metadata_to_images",
"checkpoint_downloads": "checkpoint_downloads",
"embeddings_downloads": "embeddings_downloads",
"lora_downloads": "lora_downloads"
}
REWRITE_PRESET = False
@ -366,7 +527,7 @@ def add_ratio(x):
default_aspect_ratio = add_ratio(default_aspect_ratio)
available_aspect_ratios = [add_ratio(x) for x in available_aspect_ratios]
available_aspect_ratios_labels = [add_ratio(x) for x in available_aspect_ratios]
# Only write config in the first launch.
@ -385,21 +546,49 @@ with open(config_example_path, "w", encoding="utf-8") as json_file:
'and there is no "," before the last "}". \n\n\n')
json.dump({k: config_dict[k] for k in visited_keys}, json_file, indent=4)
os.makedirs(path_outputs, exist_ok=True)
model_filenames = []
lora_filenames = []
lora_filenames_no_special = []
vae_filenames = []
wildcard_filenames = []
sdxl_lcm_lora = 'sdxl_lcm_lora.safetensors'
sdxl_lightning_lora = 'sdxl_lightning_4step_lora.safetensors'
sdxl_hyper_sd_lora = 'sdxl_hyper_sd_4step_lora.safetensors'
loras_metadata_remove = [sdxl_lcm_lora, sdxl_lightning_lora, sdxl_hyper_sd_lora]
def get_model_filenames(folder_path, name_filter=None):
return get_files_from_folder(folder_path, ['.pth', '.ckpt', '.bin', '.safetensors', '.fooocus.patch'], name_filter)
def remove_special_loras(lora_filenames):
global loras_metadata_remove
loras_no_special = lora_filenames.copy()
for lora_to_remove in loras_metadata_remove:
if lora_to_remove in loras_no_special:
loras_no_special.remove(lora_to_remove)
return loras_no_special
def update_all_model_names():
global model_filenames, lora_filenames
model_filenames = get_model_filenames(path_checkpoints)
lora_filenames = get_model_filenames(path_loras)
def get_model_filenames(folder_paths, extensions=None, name_filter=None):
if extensions is None:
extensions = ['.pth', '.ckpt', '.bin', '.safetensors', '.fooocus.patch']
files = []
if not isinstance(folder_paths, list):
folder_paths = [folder_paths]
for folder in folder_paths:
files += get_files_from_folder(folder, extensions, name_filter)
return files
def update_files():
global model_filenames, lora_filenames, lora_filenames_no_special, vae_filenames, wildcard_filenames, available_presets
model_filenames = get_model_filenames(paths_checkpoints)
lora_filenames = get_model_filenames(paths_loras)
lora_filenames_no_special = remove_special_loras(lora_filenames)
vae_filenames = get_model_filenames(path_vae)
wildcard_filenames = get_files_from_folder(path_wildcards, ['.txt'])
available_presets = get_presets()
return
@ -444,10 +633,27 @@ def downloading_inpaint_models(v):
def downloading_sdxl_lcm_lora():
load_file_from_url(
url='https://huggingface.co/lllyasviel/misc/resolve/main/sdxl_lcm_lora.safetensors',
model_dir=path_loras,
file_name='sdxl_lcm_lora.safetensors'
model_dir=paths_loras[0],
file_name=sdxl_lcm_lora
)
return 'sdxl_lcm_lora.safetensors'
return sdxl_lcm_lora
def downloading_sdxl_lightning_lora():
load_file_from_url(
url='https://huggingface.co/mashb1t/misc/resolve/main/sdxl_lightning_4step_lora.safetensors',
model_dir=paths_loras[0],
file_name=sdxl_lightning_lora
)
return sdxl_lightning_lora
def downloading_sdxl_hyper_sd_lora():
load_file_from_url(
url='https://huggingface.co/mashb1t/misc/resolve/main/sdxl_hyper_sd_4step_lora.safetensors',
model_dir=paths_loras[0],
file_name=sdxl_hyper_sd_lora
)
return sdxl_hyper_sd_lora
def downloading_controlnet_canny():
@ -514,5 +720,13 @@ def downloading_upscale_model():
)
return os.path.join(path_upscale_models, 'fooocus_upscaler_s409985e5.bin')
def downloading_safety_checker_model():
load_file_from_url(
url='https://huggingface.co/mashb1t/misc/resolve/main/stable-diffusion-safety-checker.bin',
model_dir=path_safety_checker,
file_name='stable-diffusion-safety-checker.bin'
)
return os.path.join(path_safety_checker, 'stable-diffusion-safety-checker.bin')
update_all_model_names()
update_files()

31
modules/core.py
View File

@ -1,8 +1,3 @@
from modules.patch import patch_all
patch_all()
import os
import einops
import torch
@ -16,7 +11,6 @@ import ldm_patched.modules.controlnet
import modules.sample_hijack
import ldm_patched.modules.samplers
import ldm_patched.modules.latent_formats
import modules.advanced_parameters
from ldm_patched.modules.sd import load_checkpoint_guess_config
from ldm_patched.contrib.external import VAEDecode, EmptyLatentImage, VAEEncode, VAEEncodeTiled, VAEDecodeTiled, \
@ -24,6 +18,7 @@ from ldm_patched.contrib.external import VAEDecode, EmptyLatentImage, VAEEncode,
from ldm_patched.contrib.external_freelunch import FreeU_V2
from ldm_patched.modules.sample import prepare_mask
from modules.lora import match_lora
from modules.util import get_file_from_folder_list
from ldm_patched.modules.lora import model_lora_keys_unet, model_lora_keys_clip
from modules.config import path_embeddings
from ldm_patched.contrib.external_model_advanced import ModelSamplingDiscrete
@ -40,12 +35,13 @@ opModelSamplingDiscrete = ModelSamplingDiscrete()
class StableDiffusionModel:
def __init__(self, unet=None, vae=None, clip=None, clip_vision=None, filename=None):
def __init__(self, unet=None, vae=None, clip=None, clip_vision=None, filename=None, vae_filename=None):
self.unet = unet
self.vae = vae
self.clip = clip
self.clip_vision = clip_vision
self.filename = filename
self.vae_filename = vae_filename
self.unet_with_lora = unet
self.clip_with_lora = clip
self.visited_loras = ''
@ -78,14 +74,14 @@ class StableDiffusionModel:
loras_to_load = []
for name, weight in loras:
if name == 'None':
for filename, weight in loras:
if filename == 'None':
continue
if os.path.exists(name):
lora_filename = name
if os.path.exists(filename):
lora_filename = filename
else:
lora_filename = os.path.join(modules.config.path_loras, name)
lora_filename = get_file_from_folder_list(filename, modules.config.paths_loras)
if not os.path.exists(lora_filename):
print(f'Lora file not found: {lora_filename}')
@ -147,9 +143,10 @@ def apply_controlnet(positive, negative, control_net, image, strength, start_per
@torch.no_grad()
@torch.inference_mode()
def load_model(ckpt_filename):
unet, clip, vae, clip_vision = load_checkpoint_guess_config(ckpt_filename, embedding_directory=path_embeddings)
return StableDiffusionModel(unet=unet, clip=clip, vae=vae, clip_vision=clip_vision, filename=ckpt_filename)
def load_model(ckpt_filename, vae_filename=None):
unet, clip, vae, vae_filename, clip_vision = load_checkpoint_guess_config(ckpt_filename, embedding_directory=path_embeddings,
vae_filename_param=vae_filename)
return StableDiffusionModel(unet=unet, clip=clip, vae=vae, clip_vision=clip_vision, filename=ckpt_filename, vae_filename=vae_filename)
@torch.no_grad()
@ -268,7 +265,7 @@ def get_previewer(model):
def ksampler(model, positive, negative, latent, seed=None, steps=30, cfg=7.0, sampler_name='dpmpp_2m_sde_gpu',
scheduler='karras', denoise=1.0, disable_noise=False, start_step=None, last_step=None,
force_full_denoise=False, callback_function=None, refiner=None, refiner_switch=-1,
previewer_start=None, previewer_end=None, sigmas=None, noise_mean=None):
previewer_start=None, previewer_end=None, sigmas=None, noise_mean=None, disable_preview=False):
if sigmas is not None:
sigmas = sigmas.clone().to(ldm_patched.modules.model_management.get_torch_device())
@ -299,7 +296,7 @@ def ksampler(model, positive, negative, latent, seed=None, steps=30, cfg=7.0, sa
def callback(step, x0, x, total_steps):
ldm_patched.modules.model_management.throw_exception_if_processing_interrupted()
y = None
if previewer is not None and not modules.advanced_parameters.disable_preview:
if previewer is not None and not disable_preview:
y = previewer(x0, previewer_start + step, previewer_end)
if callback_function is not None:
callback_function(previewer_start + step, x0, x, previewer_end, y)

55
modules/default_pipeline.py
View File

@ -3,6 +3,7 @@ import os
import torch
import modules.patch
import modules.config
import modules.flags
import ldm_patched.modules.model_management
import ldm_patched.modules.latent_formats
import modules.inpaint_worker
@ -11,6 +12,7 @@ from extras.expansion import FooocusExpansion
from ldm_patched.modules.model_base import SDXL, SDXLRefiner
from modules.sample_hijack import clip_separate
from modules.util import get_file_from_folder_list, get_enabled_loras
model_base = core.StableDiffusionModel()
@ -57,17 +59,21 @@ def assert_model_integrity():
@torch.no_grad()
@torch.inference_mode()
def refresh_base_model(name):
def refresh_base_model(name, vae_name=None):
global model_base
filename = os.path.abspath(os.path.realpath(os.path.join(modules.config.path_checkpoints, name)))
filename = get_file_from_folder_list(name, modules.config.paths_checkpoints)
if model_base.filename == filename:
vae_filename = None
if vae_name is not None and vae_name != modules.flags.default_vae:
vae_filename = get_file_from_folder_list(vae_name, modules.config.path_vae)
if model_base.filename == filename and model_base.vae_filename == vae_filename:
return
model_base = core.StableDiffusionModel()
model_base = core.load_model(filename)
model_base = core.load_model(filename, vae_filename)
print(f'Base model loaded: {model_base.filename}')
print(f'VAE loaded: {model_base.vae_filename}')
return
@ -76,7 +82,7 @@ def refresh_base_model(name):
def refresh_refiner_model(name):
global model_refiner
filename = os.path.abspath(os.path.realpath(os.path.join(modules.config.path_checkpoints, name)))
filename = get_file_from_folder_list(name, modules.config.paths_checkpoints)
if model_refiner.filename == filename:
return
@ -195,6 +201,17 @@ def clip_encode(texts, pool_top_k=1):
return [[torch.cat(cond_list, dim=1), {"pooled_output": pooled_acc}]]
@torch.no_grad()
@torch.inference_mode()
def set_clip_skip(clip_skip: int):
global final_clip
if final_clip is None:
return
final_clip.clip_layer(-abs(clip_skip))
return
@torch.no_grad()
@torch.inference_mode()
def clear_all_caches():
@ -215,7 +232,7 @@ def prepare_text_encoder(async_call=True):
@torch.no_grad()
@torch.inference_mode()
def refresh_everything(refiner_model_name, base_model_name, loras,
base_model_additional_loras=None, use_synthetic_refiner=False):
base_model_additional_loras=None, use_synthetic_refiner=False, vae_name=None):
global final_unet, final_clip, final_vae, final_refiner_unet, final_refiner_vae, final_expansion
final_unet = None
@ -226,11 +243,11 @@ def refresh_everything(refiner_model_name, base_model_name, loras,
if use_synthetic_refiner and refiner_model_name == 'None':
print('Synthetic Refiner Activated')
refresh_base_model(base_model_name)
refresh_base_model(base_model_name, vae_name)
synthesize_refiner_model()
else:
refresh_refiner_model(refiner_model_name)
refresh_base_model(base_model_name)
refresh_base_model(base_model_name, vae_name)
refresh_loras(loras, base_model_additional_loras=base_model_additional_loras)
assert_model_integrity()
@ -253,7 +270,8 @@ def refresh_everything(refiner_model_name, base_model_name, loras,
refresh_everything(
refiner_model_name=modules.config.default_refiner_model_name,
base_model_name=modules.config.default_base_model_name,
loras=modules.config.default_loras
loras=get_enabled_loras(modules.config.default_loras),
vae_name=modules.config.default_vae,
)
@ -315,7 +333,7 @@ def get_candidate_vae(steps, switch, denoise=1.0, refiner_swap_method='joint'):
@torch.no_grad()
@torch.inference_mode()
def process_diffusion(positive_cond, negative_cond, steps, switch, width, height, image_seed, callback, sampler_name, scheduler_name, latent=None, denoise=1.0, tiled=False, cfg_scale=7.0, refiner_swap_method='joint'):
def process_diffusion(positive_cond, negative_cond, steps, switch, width, height, image_seed, callback, sampler_name, scheduler_name, latent=None, denoise=1.0, tiled=False, cfg_scale=7.0, refiner_swap_method='joint', disable_preview=False):
target_unet, target_vae, target_refiner_unet, target_refiner_vae, target_clip \
= final_unet, final_vae, final_refiner_unet, final_refiner_vae, final_clip
@ -374,6 +392,7 @@ def process_diffusion(positive_cond, negative_cond, steps, switch, width, height
refiner_switch=switch,
previewer_start=0,
previewer_end=steps,
disable_preview=disable_preview
)
decoded_latent = core.decode_vae(vae=target_vae, latent_image=sampled_latent, tiled=tiled)
@ -392,6 +411,7 @@ def process_diffusion(positive_cond, negative_cond, steps, switch, width, height
scheduler=scheduler_name,
previewer_start=0,
previewer_end=steps,
disable_preview=disable_preview
)
print('Refiner swapped by changing ksampler. Noise preserved.')
@ -414,6 +434,7 @@ def process_diffusion(positive_cond, negative_cond, steps, switch, width, height
scheduler=scheduler_name,
previewer_start=switch,
previewer_end=steps,
disable_preview=disable_preview
)
target_model = target_refiner_vae
@ -422,7 +443,7 @@ def process_diffusion(positive_cond, negative_cond, steps, switch, width, height
decoded_latent = core.decode_vae(vae=target_model, latent_image=sampled_latent, tiled=tiled)
if refiner_swap_method == 'vae':
modules.patch.eps_record = 'vae'
modules.patch.patch_settings[os.getpid()].eps_record = 'vae'
if modules.inpaint_worker.current_task is not None:
modules.inpaint_worker.current_task.unswap()
@ -440,7 +461,8 @@ def process_diffusion(positive_cond, negative_cond, steps, switch, width, height
sampler_name=sampler_name,
scheduler=scheduler_name,
previewer_start=0,
previewer_end=steps
previewer_end=steps,
disable_preview=disable_preview
)
print('Fooocus VAE-based swap.')
@ -459,7 +481,7 @@ def process_diffusion(positive_cond, negative_cond, steps, switch, width, height
denoise=denoise)[switch:] * k_sigmas
len_sigmas = len(sigmas) - 1
noise_mean = torch.mean(modules.patch.eps_record, dim=1, keepdim=True)
noise_mean = torch.mean(modules.patch.patch_settings[os.getpid()].eps_record, dim=1, keepdim=True)
if modules.inpaint_worker.current_task is not None:
modules.inpaint_worker.current_task.swap()
@ -479,7 +501,8 @@ def process_diffusion(positive_cond, negative_cond, steps, switch, width, height
previewer_start=switch,
previewer_end=steps,
sigmas=sigmas,
noise_mean=noise_mean
noise_mean=noise_mean,
disable_preview=disable_preview
)
target_model = target_refiner_vae
@ -488,5 +511,5 @@ def process_diffusion(positive_cond, negative_cond, steps, switch, width, height
decoded_latent = core.decode_vae(vae=target_model, latent_image=sampled_latent, tiled=tiled)
images = core.pytorch_to_numpy(decoded_latent)
modules.patch.eps_record = None
modules.patch.patch_settings[os.getpid()].eps_record = None
return images

26
modules/extra_utils.py Normal file
View File

@ -0,0 +1,26 @@
import os
def makedirs_with_log(path):
try:
os.makedirs(path, exist_ok=True)
except OSError as error:
print(f'Directory {path} could not be created, reason: {error}')
def get_files_from_folder(folder_path, extensions=None, name_filter=None):
if not os.path.isdir(folder_path):
raise ValueError("Folder path is not a valid directory.")
filenames = []
for root, _, files in os.walk(folder_path, topdown=False):
relative_path = os.path.relpath(root, folder_path)
if relative_path == ".":
relative_path = ""
for filename in sorted(files, key=lambda s: s.casefold()):
_, file_extension = os.path.splitext(filename)
if (extensions is None or file_extension.lower() in extensions) and (name_filter is None or name_filter in _):
path = os.path.join(relative_path, filename)
filenames.append(path)
return filenames

125
modules/flags.py
View File

@ -1,3 +1,5 @@
from enum import IntEnum, Enum
disabled = 'Disabled'
enabled = 'Enabled'
subtle_variation = 'Vary (Subtle)'
@ -10,16 +12,54 @@ uov_list = [
disabled, subtle_variation, strong_variation, upscale_15, upscale_2, upscale_fast
]
KSAMPLER_NAMES = ["euler", "euler_ancestral", "heun", "heunpp2","dpm_2", "dpm_2_ancestral",
"lms", "dpm_fast", "dpm_adaptive", "dpmpp_2s_ancestral", "dpmpp_sde", "dpmpp_sde_gpu",
"dpmpp_2m", "dpmpp_2m_sde", "dpmpp_2m_sde_gpu", "dpmpp_3m_sde", "dpmpp_3m_sde_gpu", "ddpm", "lcm"]
CIVITAI_NO_KARRAS = ["euler", "euler_ancestral", "heun", "dpm_fast", "dpm_adaptive", "ddim", "uni_pc"]
SCHEDULER_NAMES = ["normal", "karras", "exponential", "sgm_uniform", "simple", "ddim_uniform", "lcm", "turbo"]
SAMPLER_NAMES = KSAMPLER_NAMES + ["ddim", "uni_pc", "uni_pc_bh2"]
# fooocus: a1111 (Civitai)
KSAMPLER = {
"euler": "Euler",
"euler_ancestral": "Euler a",
"heun": "Heun",
"heunpp2": "",
"dpm_2": "DPM2",
"dpm_2_ancestral": "DPM2 a",
"lms": "LMS",
"dpm_fast": "DPM fast",
"dpm_adaptive": "DPM adaptive",
"dpmpp_2s_ancestral": "DPM++ 2S a",
"dpmpp_sde": "DPM++ SDE",
"dpmpp_sde_gpu": "DPM++ SDE",
"dpmpp_2m": "DPM++ 2M",
"dpmpp_2m_sde": "DPM++ 2M SDE",
"dpmpp_2m_sde_gpu": "DPM++ 2M SDE",
"dpmpp_3m_sde": "",
"dpmpp_3m_sde_gpu": "",
"ddpm": "",
"lcm": "LCM",
"tcd": "TCD"
}
SAMPLER_EXTRA = {
"ddim": "DDIM",
"uni_pc": "UniPC",
"uni_pc_bh2": ""
}
SAMPLERS = KSAMPLER | SAMPLER_EXTRA
KSAMPLER_NAMES = list(KSAMPLER.keys())
SCHEDULER_NAMES = ["normal", "karras", "exponential", "sgm_uniform", "simple", "ddim_uniform", "lcm", "turbo", "align_your_steps", "tcd"]
SAMPLER_NAMES = KSAMPLER_NAMES + list(SAMPLER_EXTRA.keys())
sampler_list = SAMPLER_NAMES
scheduler_list = SCHEDULER_NAMES
clip_skip_max = 12
default_vae = 'Default (model)'
refiner_swap_method = 'joint'
cn_ip = "ImagePrompt"
cn_ip_face = "FaceSwap"
cn_canny = "PyraCanny"
@ -32,9 +72,9 @@ default_parameters = {
cn_ip: (0.5, 0.6), cn_ip_face: (0.9, 0.75), cn_canny: (0.5, 1.0), cn_cpds: (0.5, 1.0)
} # stop, weight
inpaint_engine_versions = ['None', 'v1', 'v2.5', 'v2.6']
performance_selections = ['Speed', 'Quality', 'Extreme Speed']
output_formats = ['png', 'jpeg', 'webp']
inpaint_engine_versions = ['None', 'v1', 'v2.5', 'v2.6']
inpaint_option_default = 'Inpaint or Outpaint (default)'
inpaint_option_detail = 'Improve Detail (face, hand, eyes, etc.)'
inpaint_option_modify = 'Modify Content (add objects, change background, etc.)'
@ -42,3 +82,74 @@ inpaint_options = [inpaint_option_default, inpaint_option_detail, inpaint_option
desc_type_photo = 'Photograph'
desc_type_anime = 'Art/Anime'
sdxl_aspect_ratios = [
'704*1408', '704*1344', '768*1344', '768*1280', '832*1216', '832*1152',
'896*1152', '896*1088', '960*1088', '960*1024', '1024*1024', '1024*960',
'1088*960', '1088*896', '1152*896', '1152*832', '1216*832', '1280*768',
'1344*768', '1344*704', '1408*704', '1472*704', '1536*640', '1600*640',
'1664*576', '1728*576'
]
class MetadataScheme(Enum):
FOOOCUS = 'fooocus'
A1111 = 'a1111'
metadata_scheme = [
(f'{MetadataScheme.FOOOCUS.value} (json)', MetadataScheme.FOOOCUS.value),
(f'{MetadataScheme.A1111.value} (plain text)', MetadataScheme.A1111.value),
]
controlnet_image_count = 4
preparation_step_count = 13
class OutputFormat(Enum):
PNG = 'png'
JPEG = 'jpeg'
WEBP = 'webp'
@classmethod
def list(cls) -> list:
return list(map(lambda c: c.value, cls))
class Steps(IntEnum):
QUALITY = 60
SPEED = 30
EXTREME_SPEED = 8
LIGHTNING = 4
HYPER_SD = 4
class StepsUOV(IntEnum):
QUALITY = 36
SPEED = 18
EXTREME_SPEED = 8
LIGHTNING = 4
HYPER_SD = 4
class Performance(Enum):
QUALITY = 'Quality'
SPEED = 'Speed'
EXTREME_SPEED = 'Extreme Speed'
LIGHTNING = 'Lightning'
HYPER_SD = 'Hyper-SD'
@classmethod
def list(cls) -> list:
return list(map(lambda c: c.value, cls))
@classmethod
def has_restricted_features(cls, x) -> bool:
if isinstance(x, Performance):
x = x.value
return x in [cls.EXTREME_SPEED.value, cls.LIGHTNING.value, cls.HYPER_SD.value]
def steps(self) -> int | None:
return Steps[self.name].value if Steps[self.name] else None
def steps_uov(self) -> int | None:
return StepsUOV[self.name].value if Steps[self.name] else None

7
modules/gradio_hijack.py
View File

@ -17,7 +17,7 @@ from gradio_client.documentation import document, set_documentation_group
from gradio_client.serializing import ImgSerializable
from PIL import Image as _Image # using _ to minimize namespace pollution
from gradio import processing_utils, utils
from gradio import processing_utils, utils, Error
from gradio.components.base import IOComponent, _Keywords, Block
from gradio.deprecation import warn_style_method_deprecation
from gradio.events import (
@ -275,7 +275,10 @@ class Image(
x, mask = x["image"], x["mask"]
assert isinstance(x, str)
im = processing_utils.decode_base64_to_image(x)
try:
im = processing_utils.decode_base64_to_image(x)
except PIL.UnidentifiedImageError:
raise Error("Unsupported image type in input")
with warnings.catch_warnings():
warnings.simplefilter("ignore")
im = im.convert(self.image_mode)

115
modules/html.py
View File

@ -1,118 +1,3 @@
css = '''
.loader-container {
display: flex; /* Use flex to align items horizontally */
align-items: center; /* Center items vertically within the container */
white-space: nowrap; /* Prevent line breaks within the container */
}
.loader {
border: 8px solid #f3f3f3; /* Light grey */
border-top: 8px solid #3498db; /* Blue */
border-radius: 50%;
width: 30px;
height: 30px;
animation: spin 2s linear infinite;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
/* Style the progress bar */
progress {
appearance: none; /* Remove default styling */
height: 20px; /* Set the height of the progress bar */
border-radius: 5px; /* Round the corners of the progress bar */
background-color: #f3f3f3; /* Light grey background */
width: 100%;
}
/* Style the progress bar container */
.progress-container {
margin-left: 20px;
margin-right: 20px;
flex-grow: 1; /* Allow the progress container to take up remaining space */
}
/* Set the color of the progress bar fill */
progress::-webkit-progress-value {
background-color: #3498db; /* Blue color for the fill */
}
progress::-moz-progress-bar {
background-color: #3498db; /* Blue color for the fill in Firefox */
}
/* Style the text on the progress bar */
progress::after {
content: attr(value '%'); /* Display the progress value followed by '%' */
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
color: white; /* Set text color */
font-size: 14px; /* Set font size */
}
/* Style other texts */
.loader-container > span {
margin-left: 5px; /* Add spacing between the progress bar and the text */
}
.progress-bar > .generating {
display: none !important;
}
.progress-bar{
height: 30px !important;
}
.type_row{
height: 80px !important;
}
.type_row_half{
height: 32px !important;
}
.scroll-hide{
resize: none !important;
}
.refresh_button{
border: none !important;
background: none !important;
font-size: none !important;
box-shadow: none !important;
}
.advanced_check_row{
width: 250px !important;
}
.min_check{
min-width: min(1px, 100%) !important;
}
.resizable_area {
resize: vertical;
overflow: auto !important;
}
.aspect_ratios label {
width: 140px !important;
}
.aspect_ratios label span {
white-space: nowrap !important;
}
.aspect_ratios label input {
margin-left: -5px !important;
}
'''
progress_html = '''
<div class="loader-container">
<div class="loader"></div>

22
modules/launch_util.py
View File

@ -1,6 +1,7 @@
import os
import importlib
import importlib.util
import shutil
import subprocess
import sys
import re
@ -9,13 +10,10 @@ import importlib.metadata
import packaging.version
from packaging.requirements import Requirement
logging.getLogger("torch.distributed.nn").setLevel(logging.ERROR) # sshh...
logging.getLogger("xformers").addFilter(lambda record: 'A matching Triton is not available' not in record.getMessage())
re_requirement = re.compile(r"\s*([-_a-zA-Z0-9]+)\s*(?:==\s*([-+_.a-zA-Z0-9]+))?\s*")
re_requirement = re.compile(r"\s*([-\w]+)\s*(?:==\s*([-+.\w]+))?\s*")
python = sys.executable
default_command_live = (os.environ.get('LAUNCH_LIVE_OUTPUT') == "1")
@ -101,3 +99,19 @@ def requirements_met(requirements_file):
return True
def delete_folder_content(folder, prefix=None):
result = True
for filename in os.listdir(folder):
file_path = os.path.join(folder, filename)
try:
if os.path.isfile(file_path) or os.path.islink(file_path):
os.unlink(file_path)
elif os.path.isdir(file_path):
shutil.rmtree(file_path)
except Exception as e:
print(f'{prefix}Failed to delete {file_path}. Reason: {e}')
result = False
return result

633
modules/meta_parser.py
View File

@ -1,78 +1,159 @@
import json
import re
from abc import ABC, abstractmethod
from pathlib import Path
import gradio as gr
from PIL import Image
import fooocus_version
import modules.config
import modules.sdxl_styles
from modules.flags import MetadataScheme, Performance, Steps
from modules.flags import SAMPLERS, CIVITAI_NO_KARRAS
from modules.util import quote, unquote, extract_styles_from_prompt, is_json, get_file_from_folder_list, sha256
re_param_code = r'\s*(\w[\w \-/]+):\s*("(?:\\.|[^\\"])+"|[^,]*)(?:,|$)'
re_param = re.compile(re_param_code)
re_imagesize = re.compile(r"^(\d+)x(\d+)$")
hash_cache = {}
def load_parameter_button_click(raw_prompt_txt, is_generating):
loaded_parameter_dict = json.loads(raw_prompt_txt)
def load_parameter_button_click(raw_metadata: dict | str, is_generating: bool):
loaded_parameter_dict = raw_metadata
if isinstance(raw_metadata, str):
loaded_parameter_dict = json.loads(raw_metadata)
assert isinstance(loaded_parameter_dict, dict)
results = [True, 1]
results = [len(loaded_parameter_dict) > 0]
get_image_number('image_number', 'Image Number', loaded_parameter_dict, results)
get_str('prompt', 'Prompt', loaded_parameter_dict, results)
get_str('negative_prompt', 'Negative Prompt', loaded_parameter_dict, results)
get_list('styles', 'Styles', loaded_parameter_dict, results)
get_str('performance', 'Performance', loaded_parameter_dict, results)
get_steps('steps', 'Steps', loaded_parameter_dict, results)
get_number('overwrite_switch', 'Overwrite Switch', loaded_parameter_dict, results)
get_resolution('resolution', 'Resolution', loaded_parameter_dict, results)
get_number('guidance_scale', 'Guidance Scale', loaded_parameter_dict, results)
get_number('sharpness', 'Sharpness', loaded_parameter_dict, results)
get_adm_guidance('adm_guidance', 'ADM Guidance', loaded_parameter_dict, results)
get_str('refiner_swap_method', 'Refiner Swap Method', loaded_parameter_dict, results)
get_number('adaptive_cfg', 'CFG Mimicking from TSNR', loaded_parameter_dict, results)
get_number('clip_skip', 'CLIP Skip', loaded_parameter_dict, results, cast_type=int)
get_str('base_model', 'Base Model', loaded_parameter_dict, results)
get_str('refiner_model', 'Refiner Model', loaded_parameter_dict, results)
get_number('refiner_switch', 'Refiner Switch', loaded_parameter_dict, results)
get_str('sampler', 'Sampler', loaded_parameter_dict, results)
get_str('scheduler', 'Scheduler', loaded_parameter_dict, results)
get_str('vae', 'VAE', loaded_parameter_dict, results)
get_seed('seed', 'Seed', loaded_parameter_dict, results)
if is_generating:
results.append(gr.update())
else:
results.append(gr.update(visible=True))
results.append(gr.update(visible=False))
get_freeu('freeu', 'FreeU', loaded_parameter_dict, results)
for i in range(modules.config.default_max_lora_number):
get_lora(f'lora_combined_{i + 1}', f'LoRA {i + 1}', loaded_parameter_dict, results)
return results
def get_str(key: str, fallback: str | None, source_dict: dict, results: list, default=None):
try:
h = loaded_parameter_dict.get('Prompt', None)
h = source_dict.get(key, source_dict.get(fallback, default))
assert isinstance(h, str)
results.append(h)
except:
results.append(gr.update())
try:
h = loaded_parameter_dict.get('Negative Prompt', None)
assert isinstance(h, str)
results.append(h)
except:
results.append(gr.update())
def get_list(key: str, fallback: str | None, source_dict: dict, results: list, default=None):
try:
h = loaded_parameter_dict.get('Styles', None)
h = source_dict.get(key, source_dict.get(fallback, default))
h = eval(h)
assert isinstance(h, list)
results.append(h)
except:
results.append(gr.update())
def get_number(key: str, fallback: str | None, source_dict: dict, results: list, default=None, cast_type=float):
try:
h = loaded_parameter_dict.get('Performance', None)
assert isinstance(h, str)
h = source_dict.get(key, source_dict.get(fallback, default))
assert h is not None
h = cast_type(h)
results.append(h)
except:
results.append(gr.update())
def get_image_number(key: str, fallback: str | None, source_dict: dict, results: list, default=None):
try:
h = loaded_parameter_dict.get('Resolution', None)
h = source_dict.get(key, source_dict.get(fallback, default))
assert h is not None
h = int(h)
h = min(h, modules.config.default_max_image_number)
results.append(h)
except:
results.append(1)
def get_steps(key: str, fallback: str | None, source_dict: dict, results: list, default=None):
try:
h = source_dict.get(key, source_dict.get(fallback, default))
assert h is not None
h = int(h)
# if not in steps or in steps and performance is not the same
if h not in iter(Steps) or Steps(h).name.casefold() != source_dict.get('performance', '').replace(' ',
'_').casefold():
results.append(h)
return
results.append(-1)
except:
results.append(-1)
def get_resolution(key: str, fallback: str | None, source_dict: dict, results: list, default=None):
try:
h = source_dict.get(key, source_dict.get(fallback, default))
width, height = eval(h)
formatted = modules.config.add_ratio(f'{width}*{height}')
if formatted in modules.config.available_aspect_ratios:
if formatted in modules.config.available_aspect_ratios_labels:
results.append(formatted)
results.append(-1)
results.append(-1)
else:
results.append(gr.update())
results.append(width)
results.append(height)
results.append(int(width))
results.append(int(height))
except:
results.append(gr.update())
results.append(gr.update())
results.append(gr.update())
def get_seed(key: str, fallback: str | None, source_dict: dict, results: list, default=None):
try:
h = loaded_parameter_dict.get('Sharpness', None)
h = source_dict.get(key, source_dict.get(fallback, default))
assert h is not None
h = float(h)
h = int(h)
results.append(False)
results.append(h)
except:
results.append(gr.update())
try:
h = loaded_parameter_dict.get('Guidance Scale', None)
assert h is not None
h = float(h)
results.append(h)
except:
results.append(gr.update())
def get_adm_guidance(key: str, fallback: str | None, source_dict: dict, results: list, default=None):
try:
h = loaded_parameter_dict.get('ADM Guidance', None)
h = source_dict.get(key, source_dict.get(fallback, default))
p, n, e = eval(h)
results.append(float(p))
results.append(float(n))
@ -82,67 +163,453 @@ def load_parameter_button_click(raw_prompt_txt, is_generating):
results.append(gr.update())
results.append(gr.update())
try:
h = loaded_parameter_dict.get('Base Model', None)
assert isinstance(h, str)
results.append(h)
except:
results.append(gr.update())
def get_freeu(key: str, fallback: str | None, source_dict: dict, results: list, default=None):
try:
h = loaded_parameter_dict.get('Refiner Model', None)
assert isinstance(h, str)
results.append(h)
h = source_dict.get(key, source_dict.get(fallback, default))
b1, b2, s1, s2 = eval(h)
results.append(True)
results.append(float(b1))
results.append(float(b2))
results.append(float(s1))
results.append(float(s2))
except:
results.append(gr.update())
try:
h = loaded_parameter_dict.get('Refiner Switch', None)
assert h is not None
h = float(h)
results.append(h)
except:
results.append(gr.update())
try:
h = loaded_parameter_dict.get('Sampler', None)
assert isinstance(h, str)
results.append(h)
except:
results.append(gr.update())
try:
h = loaded_parameter_dict.get('Scheduler', None)
assert isinstance(h, str)
results.append(h)
except:
results.append(gr.update())
try:
h = loaded_parameter_dict.get('Seed', None)
assert h is not None
h = int(h)
results.append(False)
results.append(h)
results.append(gr.update())
results.append(gr.update())
results.append(gr.update())
results.append(gr.update())
def get_lora(key: str, fallback: str | None, source_dict: dict, results: list):
try:
split_data = source_dict.get(key, source_dict.get(fallback)).split(' : ')
enabled = True
name = split_data[0]
weight = split_data[1]
if len(split_data) == 3:
enabled = split_data[0] == 'True'
name = split_data[1]
weight = split_data[2]
weight = float(weight)
results.append(enabled)
results.append(name)
results.append(weight)
except:
results.append(gr.update())
results.append(gr.update())
results.append(True)
results.append('None')
results.append(1)
if is_generating:
results.append(gr.update())
else:
results.append(gr.update(visible=True))
results.append(gr.update(visible=False))
for i in range(1, 6):
try:
n, w = loaded_parameter_dict.get(f'LoRA {i}').split(' : ')
w = float(w)
results.append(n)
results.append(w)
except:
results.append(gr.update())
results.append(gr.update())
def get_sha256(filepath):
global hash_cache
if filepath not in hash_cache:
hash_cache[filepath] = sha256(filepath)
return results
return hash_cache[filepath]
def parse_meta_from_preset(preset_content):
assert isinstance(preset_content, dict)
preset_prepared = {}
items = preset_content
for settings_key, meta_key in modules.config.possible_preset_keys.items():
if settings_key == "default_loras":
loras = getattr(modules.config, settings_key)
if settings_key in items:
loras = items[settings_key]
for index, lora in enumerate(loras[:5]):
preset_prepared[f'lora_combined_{index + 1}'] = ' : '.join(map(str, lora))
elif settings_key == "default_aspect_ratio":
if settings_key in items and items[settings_key] is not None:
default_aspect_ratio = items[settings_key]
width, height = default_aspect_ratio.split('*')
else:
default_aspect_ratio = getattr(modules.config, settings_key)
width, height = default_aspect_ratio.split('×')
height = height[:height.index(" ")]
preset_prepared[meta_key] = (width, height)
else:
preset_prepared[meta_key] = items[settings_key] if settings_key in items and items[
settings_key] is not None else getattr(modules.config, settings_key)
if settings_key == "default_styles" or settings_key == "default_aspect_ratio":
preset_prepared[meta_key] = str(preset_prepared[meta_key])
return preset_prepared
class MetadataParser(ABC):
def __init__(self):
self.raw_prompt: str = ''
self.full_prompt: str = ''
self.raw_negative_prompt: str = ''
self.full_negative_prompt: str = ''
self.steps: int = 30
self.base_model_name: str = ''
self.base_model_hash: str = ''
self.refiner_model_name: str = ''
self.refiner_model_hash: str = ''
self.loras: list = []
self.vae_name: str = ''
@abstractmethod
def get_scheme(self) -> MetadataScheme:
raise NotImplementedError
@abstractmethod
def parse_json(self, metadata: dict | str) -> dict:
raise NotImplementedError
@abstractmethod
def parse_string(self, metadata: dict) -> str:
raise NotImplementedError
def set_data(self, raw_prompt, full_prompt, raw_negative_prompt, full_negative_prompt, steps, base_model_name,
refiner_model_name, loras, vae_name):
self.raw_prompt = raw_prompt
self.full_prompt = full_prompt
self.raw_negative_prompt = raw_negative_prompt
self.full_negative_prompt = full_negative_prompt
self.steps = steps
self.base_model_name = Path(base_model_name).stem
base_model_path = get_file_from_folder_list(base_model_name, modules.config.paths_checkpoints)
self.base_model_hash = get_sha256(base_model_path)
if refiner_model_name not in ['', 'None']:
self.refiner_model_name = Path(refiner_model_name).stem
refiner_model_path = get_file_from_folder_list(refiner_model_name, modules.config.paths_checkpoints)
self.refiner_model_hash = get_sha256(refiner_model_path)
self.loras = []
for (lora_name, lora_weight) in loras:
if lora_name != 'None':
lora_path = get_file_from_folder_list(lora_name, modules.config.paths_loras)
lora_hash = get_sha256(lora_path)
self.loras.append((Path(lora_name).stem, lora_weight, lora_hash))
self.vae_name = Path(vae_name).stem
class A1111MetadataParser(MetadataParser):
def get_scheme(self) -> MetadataScheme:
return MetadataScheme.A1111
fooocus_to_a1111 = {
'raw_prompt': 'Raw prompt',
'raw_negative_prompt': 'Raw negative prompt',
'negative_prompt': 'Negative prompt',
'styles': 'Styles',
'performance': 'Performance',
'steps': 'Steps',
'sampler': 'Sampler',
'scheduler': 'Scheduler',
'vae': 'VAE',
'guidance_scale': 'CFG scale',
'seed': 'Seed',
'resolution': 'Size',
'sharpness': 'Sharpness',
'adm_guidance': 'ADM Guidance',
'refiner_swap_method': 'Refiner Swap Method',
'adaptive_cfg': 'Adaptive CFG',
'clip_skip': 'Clip skip',
'overwrite_switch': 'Overwrite Switch',
'freeu': 'FreeU',
'base_model': 'Model',
'base_model_hash': 'Model hash',
'refiner_model': 'Refiner',
'refiner_model_hash': 'Refiner hash',
'lora_hashes': 'Lora hashes',
'lora_weights': 'Lora weights',
'created_by': 'User',
'version': 'Version'
}
def parse_json(self, metadata: str) -> dict:
metadata_prompt = ''
metadata_negative_prompt = ''
done_with_prompt = False
*lines, lastline = metadata.strip().split("\n")
if len(re_param.findall(lastline)) < 3:
lines.append(lastline)
lastline = ''
for line in lines:
line = line.strip()
if line.startswith(f"{self.fooocus_to_a1111['negative_prompt']}:"):
done_with_prompt = True
line = line[len(f"{self.fooocus_to_a1111['negative_prompt']}:"):].strip()
if done_with_prompt:
metadata_negative_prompt += ('' if metadata_negative_prompt == '' else "\n") + line
else:
metadata_prompt += ('' if metadata_prompt == '' else "\n") + line
found_styles, prompt, negative_prompt = extract_styles_from_prompt(metadata_prompt, metadata_negative_prompt)
data = {
'prompt': prompt,
'negative_prompt': negative_prompt
}
for k, v in re_param.findall(lastline):
try:
if v != '' and v[0] == '"' and v[-1] == '"':
v = unquote(v)
m = re_imagesize.match(v)
if m is not None:
data['resolution'] = str((m.group(1), m.group(2)))
else:
data[list(self.fooocus_to_a1111.keys())[list(self.fooocus_to_a1111.values()).index(k)]] = v
except Exception:
print(f"Error parsing \"{k}: {v}\"")
# workaround for multiline prompts
if 'raw_prompt' in data:
data['prompt'] = data['raw_prompt']
raw_prompt = data['raw_prompt'].replace("\n", ', ')
if metadata_prompt != raw_prompt and modules.sdxl_styles.fooocus_expansion not in found_styles:
found_styles.append(modules.sdxl_styles.fooocus_expansion)
if 'raw_negative_prompt' in data:
data['negative_prompt'] = data['raw_negative_prompt']
data['styles'] = str(found_styles)
# try to load performance based on steps, fallback for direct A1111 imports
if 'steps' in data and 'performance' not in data:
try:
data['performance'] = Performance[Steps(int(data['steps'])).name].value
except ValueError | KeyError:
pass
if 'sampler' in data:
data['sampler'] = data['sampler'].replace(' Karras', '')
# get key
for k, v in SAMPLERS.items():
if v == data['sampler']:
data['sampler'] = k
break
for key in ['base_model', 'refiner_model', 'vae']:
if key in data:
if key == 'vae':
self.add_extension_to_filename(data, modules.config.vae_filenames, 'vae')
else:
self.add_extension_to_filename(data, modules.config.model_filenames, key)
lora_data = ''
if 'lora_weights' in data and data['lora_weights'] != '':
lora_data = data['lora_weights']
elif 'lora_hashes' in data and data['lora_hashes'] != '' and data['lora_hashes'].split(', ')[0].count(':') == 2:
lora_data = data['lora_hashes']
if lora_data != '':
for li, lora in enumerate(lora_data.split(', ')):
lora_split = lora.split(': ')
lora_name = lora_split[0]
lora_weight = lora_split[2] if len(lora_split) == 3 else lora_split[1]
for filename in modules.config.lora_filenames_no_special:
path = Path(filename)
if lora_name == path.stem:
data[f'lora_combined_{li + 1}'] = f'{filename} : {lora_weight}'
break
return data
def parse_string(self, metadata: dict) -> str:
data = {k: v for _, k, v in metadata}
width, height = eval(data['resolution'])
sampler = data['sampler']
scheduler = data['scheduler']
if sampler in SAMPLERS and SAMPLERS[sampler] != '':
sampler = SAMPLERS[sampler]
if sampler not in CIVITAI_NO_KARRAS and scheduler == 'karras':
sampler += f' Karras'
generation_params = {
self.fooocus_to_a1111['steps']: self.steps,
self.fooocus_to_a1111['sampler']: sampler,
self.fooocus_to_a1111['seed']: data['seed'],
self.fooocus_to_a1111['resolution']: f'{width}x{height}',
self.fooocus_to_a1111['guidance_scale']: data['guidance_scale'],
self.fooocus_to_a1111['sharpness']: data['sharpness'],
self.fooocus_to_a1111['adm_guidance']: data['adm_guidance'],
self.fooocus_to_a1111['base_model']: Path(data['base_model']).stem,
self.fooocus_to_a1111['base_model_hash']: self.base_model_hash,
self.fooocus_to_a1111['performance']: data['performance'],
self.fooocus_to_a1111['scheduler']: scheduler,
self.fooocus_to_a1111['vae']: Path(data['vae']).stem,
# workaround for multiline prompts
self.fooocus_to_a1111['raw_prompt']: self.raw_prompt,
self.fooocus_to_a1111['raw_negative_prompt']: self.raw_negative_prompt,
}
if self.refiner_model_name not in ['', 'None']:
generation_params |= {
self.fooocus_to_a1111['refiner_model']: self.refiner_model_name,
self.fooocus_to_a1111['refiner_model_hash']: self.refiner_model_hash
}
for key in ['adaptive_cfg', 'clip_skip', 'overwrite_switch', 'refiner_swap_method', 'freeu']:
if key in data:
generation_params[self.fooocus_to_a1111[key]] = data[key]
if len(self.loras) > 0:
lora_hashes = []
lora_weights = []
for index, (lora_name, lora_weight, lora_hash) in enumerate(self.loras):
# workaround for Fooocus not knowing LoRA name in LoRA metadata
lora_hashes.append(f'{lora_name}: {lora_hash}')
lora_weights.append(f'{lora_name}: {lora_weight}')
lora_hashes_string = ', '.join(lora_hashes)
lora_weights_string = ', '.join(lora_weights)
generation_params[self.fooocus_to_a1111['lora_hashes']] = lora_hashes_string
generation_params[self.fooocus_to_a1111['lora_weights']] = lora_weights_string
generation_params[self.fooocus_to_a1111['version']] = data['version']
if modules.config.metadata_created_by != '':
generation_params[self.fooocus_to_a1111['created_by']] = modules.config.metadata_created_by
generation_params_text = ", ".join(
[k if k == v else f'{k}: {quote(v)}' for k, v in generation_params.items() if
v is not None])
positive_prompt_resolved = ', '.join(self.full_prompt)
negative_prompt_resolved = ', '.join(self.full_negative_prompt)
negative_prompt_text = f"\nNegative prompt: {negative_prompt_resolved}" if negative_prompt_resolved else ""
return f"{positive_prompt_resolved}{negative_prompt_text}\n{generation_params_text}".strip()
@staticmethod
def add_extension_to_filename(data, filenames, key):
for filename in filenames:
path = Path(filename)
if data[key] == path.stem:
data[key] = filename
break
class FooocusMetadataParser(MetadataParser):
def get_scheme(self) -> MetadataScheme:
return MetadataScheme.FOOOCUS
def parse_json(self, metadata: dict) -> dict:
for key, value in metadata.items():
if value in ['', 'None']:
continue
if key in ['base_model', 'refiner_model']:
metadata[key] = self.replace_value_with_filename(key, value, modules.config.model_filenames)
elif key.startswith('lora_combined_'):
metadata[key] = self.replace_value_with_filename(key, value, modules.config.lora_filenames_no_special)
elif key == 'vae':
metadata[key] = self.replace_value_with_filename(key, value, modules.config.vae_filenames)
else:
continue
return metadata
def parse_string(self, metadata: list) -> str:
for li, (label, key, value) in enumerate(metadata):
# remove model folder paths from metadata
if key.startswith('lora_combined_'):
name, weight = value.split(' : ')
name = Path(name).stem
value = f'{name} : {weight}'
metadata[li] = (label, key, value)
res = {k: v for _, k, v in metadata}
res['full_prompt'] = self.full_prompt
res['full_negative_prompt'] = self.full_negative_prompt
res['steps'] = self.steps
res['base_model'] = self.base_model_name
res['base_model_hash'] = self.base_model_hash
if self.refiner_model_name not in ['', 'None']:
res['refiner_model'] = self.refiner_model_name
res['refiner_model_hash'] = self.refiner_model_hash
res['vae'] = self.vae_name
res['loras'] = self.loras
if modules.config.metadata_created_by != '':
res['created_by'] = modules.config.metadata_created_by
return json.dumps(dict(sorted(res.items())))
@staticmethod
def replace_value_with_filename(key, value, filenames):
for filename in filenames:
path = Path(filename)
if key.startswith('lora_combined_'):
name, weight = value.split(' : ')
if name == path.stem:
return f'{filename} : {weight}'
elif value == path.stem:
return filename
def get_metadata_parser(metadata_scheme: MetadataScheme) -> MetadataParser:
match metadata_scheme:
case MetadataScheme.FOOOCUS:
return FooocusMetadataParser()
case MetadataScheme.A1111:
return A1111MetadataParser()
case _:
raise NotImplementedError
def read_info_from_image(filepath) -> tuple[str | None, MetadataScheme | None]:
with Image.open(filepath) as image:
items = (image.info or {}).copy()
parameters = items.pop('parameters', None)
metadata_scheme = items.pop('fooocus_scheme', None)
exif = items.pop('exif', None)
if parameters is not None and is_json(parameters):
parameters = json.loads(parameters)
elif exif is not None:
exif = image.getexif()
# 0x9286 = UserComment
parameters = exif.get(0x9286, None)
# 0x927C = MakerNote
metadata_scheme = exif.get(0x927C, None)
if is_json(parameters):
parameters = json.loads(parameters)
try:
metadata_scheme = MetadataScheme(metadata_scheme)
except ValueError:
metadata_scheme = None
# broad fallback
if isinstance(parameters, dict):
metadata_scheme = MetadataScheme.FOOOCUS
if isinstance(parameters, str):
metadata_scheme = MetadataScheme.A1111
return parameters, metadata_scheme
def get_exif(metadata: str | None, metadata_scheme: str):
exif = Image.Exif()
# tags see see https://github.com/python-pillow/Pillow/blob/9.2.x/src/PIL/ExifTags.py
# 0x9286 = UserComment
exif[0x9286] = metadata
# 0x0131 = Software
exif[0x0131] = 'Fooocus v' + fooocus_version.version
# 0x927C = MakerNote
exif[0x927C] = metadata_scheme
return exif

2
modules/model_loader.py
View File

@ -14,6 +14,8 @@ def load_file_from_url(
Returns the path to the downloaded file.
"""
domain = os.environ.get("HF_MIRROR", "https://huggingface.co").rstrip('/')
url = str.replace(url, "https://huggingface.co", domain, 1)
os.makedirs(model_dir, exist_ok=True)
if not file_name:
parts = urlparse(url)

70
modules/patch.py
View File

@ -17,7 +17,6 @@ import ldm_patched.controlnet.cldm
import ldm_patched.modules.model_patcher
import ldm_patched.modules.samplers
import ldm_patched.modules.args_parser
import modules.advanced_parameters as advanced_parameters
import warnings
import safetensors.torch
import modules.constants as constants
@ -29,15 +28,25 @@ from modules.patch_precision import patch_all_precision
from modules.patch_clip import patch_all_clip
sharpness = 2.0
class PatchSettings:
def __init__(self,
sharpness=2.0,
adm_scaler_end=0.3,
positive_adm_scale=1.5,
negative_adm_scale=0.8,
controlnet_softness=0.25,
adaptive_cfg=7.0):
self.sharpness = sharpness
self.adm_scaler_end = adm_scaler_end
self.positive_adm_scale = positive_adm_scale
self.negative_adm_scale = negative_adm_scale
self.controlnet_softness = controlnet_softness
self.adaptive_cfg = adaptive_cfg
self.global_diffusion_progress = 0
self.eps_record = None
adm_scaler_end = 0.3
positive_adm_scale = 1.5
negative_adm_scale = 0.8
adaptive_cfg = 7.0
global_diffusion_progress = 0
eps_record = None
patch_settings = {}
def calculate_weight_patched(self, patches, weight, key):
@ -201,14 +210,13 @@ class BrownianTreeNoiseSamplerPatched:
def compute_cfg(uncond, cond, cfg_scale, t):
global adaptive_cfg
mimic_cfg = float(adaptive_cfg)
pid = os.getpid()
mimic_cfg = float(patch_settings[pid].adaptive_cfg)
real_cfg = float(cfg_scale)
real_eps = uncond + real_cfg * (cond - uncond)
if cfg_scale > adaptive_cfg:
if cfg_scale > patch_settings[pid].adaptive_cfg:
mimicked_eps = uncond + mimic_cfg * (cond - uncond)
return real_eps * t + mimicked_eps * (1 - t)
else:
@ -216,13 +224,13 @@ def compute_cfg(uncond, cond, cfg_scale, t):
def patched_sampling_function(model, x, timestep, uncond, cond, cond_scale, model_options=None, seed=None):
global eps_record
pid = os.getpid()
if math.isclose(cond_scale, 1.0) and not model_options.get("disable_cfg1_optimization", False):
final_x0 = calc_cond_uncond_batch(model, cond, None, x, timestep, model_options)[0]
if eps_record is not None:
eps_record = ((x - final_x0) / timestep).cpu()
if patch_settings[pid].eps_record is not None:
patch_settings[pid].eps_record = ((x - final_x0) / timestep).cpu()
return final_x0
@ -231,16 +239,16 @@ def patched_sampling_function(model, x, timestep, uncond, cond, cond_scale, mode
positive_eps = x - positive_x0
negative_eps = x - negative_x0
alpha = 0.001 * sharpness * global_diffusion_progress
alpha = 0.001 * patch_settings[pid].sharpness * patch_settings[pid].global_diffusion_progress
positive_eps_degraded = anisotropic.adaptive_anisotropic_filter(x=positive_eps, g=positive_x0)
positive_eps_degraded_weighted = positive_eps_degraded * alpha + positive_eps * (1.0 - alpha)
final_eps = compute_cfg(uncond=negative_eps, cond=positive_eps_degraded_weighted,
cfg_scale=cond_scale, t=global_diffusion_progress)
cfg_scale=cond_scale, t=patch_settings[pid].global_diffusion_progress)
if eps_record is not None:
eps_record = (final_eps / timestep).cpu()
if patch_settings[pid].eps_record is not None:
patch_settings[pid].eps_record = (final_eps / timestep).cpu()
return x - final_eps
@ -255,20 +263,19 @@ def round_to_64(x):
def sdxl_encode_adm_patched(self, **kwargs):
global positive_adm_scale, negative_adm_scale
clip_pooled = ldm_patched.modules.model_base.sdxl_pooled(kwargs, self.noise_augmentor)
width = kwargs.get("width", 1024)
height = kwargs.get("height", 1024)
target_width = width
target_height = height
pid = os.getpid()
if kwargs.get("prompt_type", "") == "negative":
width = float(width) * negative_adm_scale
height = float(height) * negative_adm_scale
width = float(width) * patch_settings[pid].negative_adm_scale
height = float(height) * patch_settings[pid].negative_adm_scale
elif kwargs.get("prompt_type", "") == "positive":
width = float(width) * positive_adm_scale
height = float(height) * positive_adm_scale
width = float(width) * patch_settings[pid].positive_adm_scale
height = float(height) * patch_settings[pid].positive_adm_scale
def embedder(number_list):
h = self.embedder(torch.tensor(number_list, dtype=torch.float32))
@ -322,7 +329,7 @@ def patched_KSamplerX0Inpaint_forward(self, x, sigma, uncond, cond, cond_scale,
def timed_adm(y, timesteps):
if isinstance(y, torch.Tensor) and int(y.dim()) == 2 and int(y.shape[1]) == 5632:
y_mask = (timesteps > 999.0 * (1.0 - float(adm_scaler_end))).to(y)[..., None]
y_mask = (timesteps > 999.0 * (1.0 - float(patch_settings[os.getpid()].adm_scaler_end))).to(y)[..., None]
y_with_adm = y[..., :2816].clone()
y_without_adm = y[..., 2816:].clone()
return y_with_adm * y_mask + y_without_adm * (1.0 - y_mask)
@ -332,6 +339,7 @@ def timed_adm(y, timesteps):
def patched_cldm_forward(self, x, hint, timesteps, context, y=None, **kwargs):
t_emb = ldm_patched.ldm.modules.diffusionmodules.openaimodel.timestep_embedding(timesteps, self.model_channels, repeat_only=False).to(x.dtype)
emb = self.time_embed(t_emb)
pid = os.getpid()
guided_hint = self.input_hint_block(hint, emb, context)
@ -357,19 +365,17 @@ def patched_cldm_forward(self, x, hint, timesteps, context, y=None, **kwargs):
h = self.middle_block(h, emb, context)
outs.append(self.middle_block_out(h, emb, context))
if advanced_parameters.controlnet_softness > 0:
if patch_settings[pid].controlnet_softness > 0:
for i in range(10):
k = 1.0 - float(i) / 9.0
outs[i] = outs[i] * (1.0 - advanced_parameters.controlnet_softness * k)
outs[i] = outs[i] * (1.0 - patch_settings[pid].controlnet_softness * k)
return outs
def patched_unet_forward(self, x, timesteps=None, context=None, y=None, control=None, transformer_options={}, **kwargs):
global global_diffusion_progress
self.current_step = 1.0 - timesteps.to(x) / 999.0
global_diffusion_progress = float(self.current_step.detach().cpu().numpy().tolist()[0])
patch_settings[os.getpid()].global_diffusion_progress = float(self.current_step.detach().cpu().numpy().tolist()[0])
y = timed_adm(y, timesteps)
@ -483,7 +489,7 @@ def patch_all():
if ldm_patched.modules.model_management.directml_enabled:
ldm_patched.modules.model_management.lowvram_available = True
ldm_patched.modules.model_management.OOM_EXCEPTION = Exception
patch_all_precision()
patch_all_clip()

2
modules/patch_precision.py
View File

@ -51,6 +51,8 @@ def patched_register_schedule(self, given_betas=None, beta_schedule="linear", ti
self.linear_end = linear_end
sigmas = torch.tensor(((1 - alphas_cumprod) / alphas_cumprod) ** 0.5, dtype=torch.float32)
self.set_sigmas(sigmas)
alphas_cumprod = torch.tensor(alphas_cumprod, dtype=torch.float32)
self.set_alphas_cumprod(alphas_cumprod)
return

61
modules/private_logger.py
View File

@ -5,26 +5,49 @@ import json
import urllib.parse
from PIL import Image
from PIL.PngImagePlugin import PngInfo
from modules.flags import OutputFormat
from modules.meta_parser import MetadataParser, get_exif
from modules.util import generate_temp_filename
log_cache = {}
def get_current_html_path():
def get_current_html_path(output_format=None):
output_format = output_format if output_format else modules.config.default_output_format
date_string, local_temp_filename, only_name = generate_temp_filename(folder=modules.config.path_outputs,
extension='png')
extension=output_format)
html_name = os.path.join(os.path.dirname(local_temp_filename), 'log.html')
return html_name
def log(img, dic):
if args_manager.args.disable_image_log:
return
date_string, local_temp_filename, only_name = generate_temp_filename(folder=modules.config.path_outputs, extension='png')
def log(img, metadata, metadata_parser: MetadataParser | None = None, output_format=None, task=None) -> str:
path_outputs = modules.config.temp_path if args_manager.args.disable_image_log else modules.config.path_outputs
output_format = output_format if output_format else modules.config.default_output_format
date_string, local_temp_filename, only_name = generate_temp_filename(folder=path_outputs, extension=output_format)
os.makedirs(os.path.dirname(local_temp_filename), exist_ok=True)
Image.fromarray(img).save(local_temp_filename)
parsed_parameters = metadata_parser.parse_string(metadata.copy()) if metadata_parser is not None else ''
image = Image.fromarray(img)
if output_format == OutputFormat.PNG.value:
if parsed_parameters != '':
pnginfo = PngInfo()
pnginfo.add_text('parameters', parsed_parameters)
pnginfo.add_text('fooocus_scheme', metadata_parser.get_scheme().value)
else:
pnginfo = None
image.save(local_temp_filename, pnginfo=pnginfo)
elif output_format == OutputFormat.JPEG.value:
image.save(local_temp_filename, quality=95, optimize=True, progressive=True, exif=get_exif(parsed_parameters, metadata_parser.get_scheme().value) if metadata_parser else Image.Exif())
elif output_format == OutputFormat.WEBP.value:
image.save(local_temp_filename, quality=95, lossless=False, exif=get_exif(parsed_parameters, metadata_parser.get_scheme().value) if metadata_parser else Image.Exif())
else:
image.save(local_temp_filename)
if args_manager.args.disable_image_log:
return local_temp_filename
html_name = os.path.join(os.path.dirname(local_temp_filename), 'log.html')
css_styles = (
@ -32,7 +55,7 @@ def log(img, dic):
"body { background-color: #121212; color: #E0E0E0; } "
"a { color: #BB86FC; } "
".metadata { border-collapse: collapse; width: 100%; } "
".metadata .key { width: 15%; } "
".metadata .label { width: 15%; } "
".metadata .value { width: 85%; font-weight: bold; } "
".metadata th, .metadata td { border: 1px solid #4d4d4d; padding: 4px; } "
".image-container img { height: auto; max-width: 512px; display: block; padding-right:10px; } "
@ -68,7 +91,7 @@ def log(img, dic):
</script>"""
)
begin_part = f"<html><head><title>Fooocus Log {date_string}</title>{css_styles}</head><body>{js}<p>Fooocus Log {date_string} (private)</p>\n<p>All images are clean, without any hidden data/meta, and safe to share with others.</p><!--fooocus-log-split-->\n\n"
begin_part = f"<!DOCTYPE html><html><head><title>Fooocus Log {date_string}</title>{css_styles}</head><body>{js}<p>Fooocus Log {date_string} (private)</p>\n<p>Metadata is embedded if enabled in the config or developer debug mode. You can find the information for each image in line Metadata Scheme.</p><!--fooocus-log-split-->\n\n"
end_part = f'\n<!--fooocus-log-split--></body></html>'
middle_part = log_cache.get(html_name, "")
@ -83,14 +106,20 @@ def log(img, dic):
div_name = only_name.replace('.', '_')
item = f"<div id=\"{div_name}\" class=\"image-container\"><hr><table><tr>\n"
item += f"<td><a href=\"{only_name}\" target=\"_blank\"><img src='{only_name}' onerror=\"this.closest('.image-container').style.display='none';\" loading='lazy'></img></a><div>{only_name}</div></td>"
item += f"<td><a href=\"{only_name}\" target=\"_blank\"><img src='{only_name}' onerror=\"this.closest('.image-container').style.display='none';\" loading='lazy'/></a><div>{only_name}</div></td>"
item += "<td><table class='metadata'>"
for key, value in dic:
for label, key, value in metadata:
value_txt = str(value).replace('\n', ' </br> ')
item += f"<tr><td class='key'>{key}</td><td class='value'>{value_txt}</td></tr>\n"
item += f"<tr><td class='label'>{label}</td><td class='value'>{value_txt}</td></tr>\n"
if task is not None and 'positive' in task and 'negative' in task:
full_prompt_details = f"""<details><summary>Positive</summary>{', '.join(task['positive'])}</details>
<details><summary>Negative</summary>{', '.join(task['negative'])}</details>"""
item += f"<tr><td class='label'>Full raw prompt</td><td class='value'>{full_prompt_details}</td></tr>\n"
item += "</table>"
js_txt = urllib.parse.quote(json.dumps({k: v for k, v in dic}, indent=0), safe='')
js_txt = urllib.parse.quote(json.dumps({k: v for _, k, v, in metadata}, indent=0), safe='')
item += f"</br><button onclick=\"to_clipboard('{js_txt}')\">Copy to Clipboard</button>"
item += "</td>"
@ -105,4 +134,4 @@ def log(img, dic):
log_cache[html_name] = middle_part
return
return local_temp_filename

4
modules/sample_hijack.py
View File

@ -3,6 +3,7 @@ import ldm_patched.modules.samplers
import ldm_patched.modules.model_management
from collections import namedtuple
from ldm_patched.contrib.external_align_your_steps import AlignYourStepsScheduler
from ldm_patched.contrib.external_custom_sampler import SDTurboScheduler
from ldm_patched.k_diffusion import sampling as k_diffusion_sampling
from ldm_patched.modules.samplers import normal_scheduler, simple_scheduler, ddim_scheduler
@ -175,6 +176,9 @@ def calculate_sigmas_scheduler_hacked(model, scheduler_name, steps):
sigmas = normal_scheduler(model, steps, sgm=True)
elif scheduler_name == "turbo":
sigmas = SDTurboScheduler().get_sigmas(namedtuple('Patcher', ['model'])(model=model), steps=steps, denoise=1.0)[0]
elif scheduler_name == "align_your_steps":
model_type = 'SDXL' if isinstance(model.latent_format, ldm_patched.modules.latent_formats.SDXL) else 'SD1'
sigmas = AlignYourStepsScheduler().get_sigmas(model_type=model_type, steps=steps, denoise=1.0)[0]
else:
raise TypeError("error invalid scheduler")
return sigmas

67
modules/sdxl_styles.py
View File

@ -1,14 +1,13 @@
import os
import re
import json
import math
from modules.util import get_files_from_folder
from modules.extra_utils import get_files_from_folder
from random import Random
# cannot use modules.config - validators causing circular imports
styles_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '../sdxl_styles/'))
wildcards_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '../wildcards/'))
wildcards_max_bfs_depth = 64
def normalize_key(k):
@ -24,7 +23,6 @@ def normalize_key(k):
styles = {}
styles_files = get_files_from_folder(styles_path, ['.json'])
for x in ['sdxl_styles_fooocus.json',
@ -50,8 +48,13 @@ for styles_file in styles_files:
print(f'Failed to load style file {styles_file}')
style_keys = list(styles.keys())
fooocus_expansion = "Fooocus V2"
legal_style_names = [fooocus_expansion] + style_keys
fooocus_expansion = 'Fooocus V2'
random_style_name = 'Random Style'
legal_style_names = [fooocus_expansion, random_style_name] + style_keys
def get_random_style(rng: Random) -> str:
return rng.choice(list(styles.items()))[0]
def apply_style(style, positive):
@ -59,24 +62,36 @@ def apply_style(style, positive):
return p.replace('{prompt}', positive).splitlines(), n.splitlines()
def apply_wildcards(wildcard_text, rng, directory=wildcards_path):
for _ in range(wildcards_max_bfs_depth):
placeholders = re.findall(r'__([\w-]+)__', wildcard_text)
if len(placeholders) == 0:
return wildcard_text
def get_words(arrays, total_mult, index):
if len(arrays) == 1:
return [arrays[0].split(',')[index]]
else:
words = arrays[0].split(',')
word = words[index % len(words)]
index -= index % len(words)
index /= len(words)
index = math.floor(index)
return [word] + get_words(arrays[1:], math.floor(total_mult / len(words)), index)
print(f'[Wildcards] processing: {wildcard_text}')
for placeholder in placeholders:
try:
words = open(os.path.join(directory, f'{placeholder}.txt'), encoding='utf-8').read().splitlines()
words = [x for x in words if x != '']
assert len(words) > 0
wildcard_text = wildcard_text.replace(f'__{placeholder}__', rng.choice(words), 1)
except:
print(f'[Wildcards] Warning: {placeholder}.txt missing or empty. '
f'Using "{placeholder}" as a normal word.')
wildcard_text = wildcard_text.replace(f'__{placeholder}__', placeholder)
print(f'[Wildcards] {wildcard_text}')
print(f'[Wildcards] BFS stack overflow. Current text: {wildcard_text}')
return wildcard_text
def apply_arrays(text, index):
arrays = re.findall(r'\[\[(.*?)\]\]', text)
if len(arrays) == 0:
return text
print(f'[Arrays] processing: {text}')
mult = 1
for arr in arrays:
words = arr.split(',')
mult *= len(words)
index %= mult
chosen_words = get_words(arrays, mult, index)
i = 0
for arr in arrays:
text = text.replace(f'[[{arr}]]', chosen_words[i], 1)
i = i+1
return text

2
modules/ui_gradio_extensions.py
View File

@ -39,7 +39,7 @@ def javascript_html():
head += f'<script type="text/javascript" src="{edit_attention_js_path}"></script>\n'
head += f'<script type="text/javascript" src="{viewer_js_path}"></script>\n'
head += f'<script type="text/javascript" src="{image_viewer_js_path}"></script>\n'
head += f'<meta name="samples-path" content="{samples_path}"></meta>\n'
head += f'<meta name="samples-path" content="{samples_path}">\n'
if args_manager.args.theme:
head += f'<script type="text/javascript">set_theme(\"{args_manager.args.theme}\");</script>\n'

355
modules/util.py
View File

@ -1,15 +1,31 @@
from pathlib import Path
import numpy as np
import datetime
import random
import math
import os
import cv2
import re
from typing import List, Tuple, AnyStr, NamedTuple
import json
import hashlib
from PIL import Image
import modules.config
import modules.sdxl_styles
LANCZOS = (Image.Resampling.LANCZOS if hasattr(Image, 'Resampling') else Image.LANCZOS)
# Regexp compiled once. Matches entries with the following pattern:
# <lora:some_lora:1>
# <lora:aNotherLora:-1.6>
LORAS_PROMPT_PATTERN = re.compile(r"(<lora:([^:]+):([+-]?(?:\d+(?:\.\d*)?|\.\d+))>)", re.X)
HASH_SHA256_LENGTH = 10
def erode_or_dilate(x, k):
k = int(k)
@ -155,23 +171,332 @@ def generate_temp_filename(folder='./outputs/', extension='png'):
random_number = random.randint(1000, 9999)
filename = f"{time_string}_{random_number}.{extension}"
result = os.path.join(folder, date_string, filename)
return date_string, os.path.abspath(os.path.realpath(result)), filename
return date_string, os.path.abspath(result), filename
def get_files_from_folder(folder_path, exensions=None, name_filter=None):
if not os.path.isdir(folder_path):
raise ValueError("Folder path is not a valid directory.")
def sha256(filename, use_addnet_hash=False, length=HASH_SHA256_LENGTH):
print(f"Calculating sha256 for {filename}: ", end='')
if use_addnet_hash:
with open(filename, "rb") as file:
sha256_value = addnet_hash_safetensors(file)
else:
sha256_value = calculate_sha256(filename)
print(f"{sha256_value}")
filenames = []
return sha256_value[:length] if length is not None else sha256_value
for root, dirs, files in os.walk(folder_path):
relative_path = os.path.relpath(root, folder_path)
if relative_path == ".":
relative_path = ""
for filename in files:
_, file_extension = os.path.splitext(filename)
if (exensions == None or file_extension.lower() in exensions) and (name_filter == None or name_filter in _):
path = os.path.join(relative_path, filename)
filenames.append(path)
return sorted(filenames, key=lambda x: -1 if os.sep in x else 1)
def addnet_hash_safetensors(b):
"""kohya-ss hash for safetensors from https://github.com/kohya-ss/sd-scripts/blob/main/library/train_util.py"""
hash_sha256 = hashlib.sha256()
blksize = 1024 * 1024
b.seek(0)
header = b.read(8)
n = int.from_bytes(header, "little")
offset = n + 8
b.seek(offset)
for chunk in iter(lambda: b.read(blksize), b""):
hash_sha256.update(chunk)
return hash_sha256.hexdigest()
def calculate_sha256(filename) -> str:
hash_sha256 = hashlib.sha256()
blksize = 1024 * 1024
with open(filename, "rb") as f:
for chunk in iter(lambda: f.read(blksize), b""):
hash_sha256.update(chunk)
return hash_sha256.hexdigest()
def quote(text):
if ',' not in str(text) and '\n' not in str(text) and ':' not in str(text):
return text
return json.dumps(text, ensure_ascii=False)
def unquote(text):
if len(text) == 0 or text[0] != '"' or text[-1] != '"':
return text
try:
return json.loads(text)
except Exception:
return text
def unwrap_style_text_from_prompt(style_text, prompt):
"""
Checks the prompt to see if the style text is wrapped around it. If so,
returns True plus the prompt text without the style text. Otherwise, returns
False with the original prompt.
Note that the "cleaned" version of the style text is only used for matching
purposes here. It isn't returned; the original style text is not modified.
"""
stripped_prompt = prompt
stripped_style_text = style_text
if "{prompt}" in stripped_style_text:
# Work out whether the prompt is wrapped in the style text. If so, we
# return True and the "inner" prompt text that isn't part of the style.
try:
left, right = stripped_style_text.split("{prompt}", 2)
except ValueError as e:
# If the style text has multple "{prompt}"s, we can't split it into
# two parts. This is an error, but we can't do anything about it.
print(f"Unable to compare style text to prompt:\n{style_text}")
print(f"Error: {e}")
return False, prompt, ''
left_pos = stripped_prompt.find(left)
right_pos = stripped_prompt.find(right)
if 0 <= left_pos < right_pos:
real_prompt = stripped_prompt[left_pos + len(left):right_pos]
prompt = stripped_prompt.replace(left + real_prompt + right, '', 1)
if prompt.startswith(", "):
prompt = prompt[2:]
if prompt.endswith(", "):
prompt = prompt[:-2]
return True, prompt, real_prompt
else:
# Work out whether the given prompt starts with the style text. If so, we
# return True and the prompt text up to where the style text starts.
if stripped_prompt.endswith(stripped_style_text):
prompt = stripped_prompt[: len(stripped_prompt) - len(stripped_style_text)]
if prompt.endswith(", "):
prompt = prompt[:-2]
return True, prompt, prompt
return False, prompt, ''
def extract_original_prompts(style, prompt, negative_prompt):
"""
Takes a style and compares it to the prompt and negative prompt. If the style
matches, returns True plus the prompt and negative prompt with the style text
removed. Otherwise, returns False with the original prompt and negative prompt.
"""
if not style.prompt and not style.negative_prompt:
return False, prompt, negative_prompt
match_positive, extracted_positive, real_prompt = unwrap_style_text_from_prompt(
style.prompt, prompt
)
if not match_positive:
return False, prompt, negative_prompt, ''
match_negative, extracted_negative, _ = unwrap_style_text_from_prompt(
style.negative_prompt, negative_prompt
)
if not match_negative:
return False, prompt, negative_prompt, ''
return True, extracted_positive, extracted_negative, real_prompt
def extract_styles_from_prompt(prompt, negative_prompt):
extracted = []
applicable_styles = []
for style_name, (style_prompt, style_negative_prompt) in modules.sdxl_styles.styles.items():
applicable_styles.append(PromptStyle(name=style_name, prompt=style_prompt, negative_prompt=style_negative_prompt))
real_prompt = ''
while True:
found_style = None
for style in applicable_styles:
is_match, new_prompt, new_neg_prompt, new_real_prompt = extract_original_prompts(
style, prompt, negative_prompt
)
if is_match:
found_style = style
prompt = new_prompt
negative_prompt = new_neg_prompt
if real_prompt == '' and new_real_prompt != '' and new_real_prompt != prompt:
real_prompt = new_real_prompt
break
if not found_style:
break
applicable_styles.remove(found_style)
extracted.append(found_style.name)
# add prompt expansion if not all styles could be resolved
if prompt != '':
if real_prompt != '':
extracted.append(modules.sdxl_styles.fooocus_expansion)
else:
# find real_prompt when only prompt expansion is selected
first_word = prompt.split(', ')[0]
first_word_positions = [i for i in range(len(prompt)) if prompt.startswith(first_word, i)]
if len(first_word_positions) > 1:
real_prompt = prompt[:first_word_positions[-1]]
extracted.append(modules.sdxl_styles.fooocus_expansion)
if real_prompt.endswith(', '):
real_prompt = real_prompt[:-2]
return list(reversed(extracted)), real_prompt, negative_prompt
class PromptStyle(NamedTuple):
name: str
prompt: str
negative_prompt: str
def is_json(data: str) -> bool:
try:
loaded_json = json.loads(data)
assert isinstance(loaded_json, dict)
except (ValueError, AssertionError):
return False
return True
def get_filname_by_stem(lora_name, filenames: List[str]) -> str | None:
for filename in filenames:
path = Path(filename)
if lora_name == path.stem:
return filename
return None
def get_file_from_folder_list(name, folders):
if not isinstance(folders, list):
folders = [folders]
for folder in folders:
filename = os.path.abspath(os.path.realpath(os.path.join(folder, name)))
if os.path.isfile(filename):
return filename
return os.path.abspath(os.path.realpath(os.path.join(folders[0], name)))
def ordinal_suffix(number: int) -> str:
return 'th' if 10 <= number % 100 <= 20 else {1: 'st', 2: 'nd', 3: 'rd'}.get(number % 10, 'th')
def makedirs_with_log(path):
try:
os.makedirs(path, exist_ok=True)
except OSError as error:
print(f'Directory {path} could not be created, reason: {error}')
def get_enabled_loras(loras: list, remove_none=True) -> list:
return [(lora[1], lora[2]) for lora in loras if lora[0] and (lora[1] != 'None' if remove_none else True)]
def parse_lora_references_from_prompt(prompt: str, loras: List[Tuple[AnyStr, float]], loras_limit: int = 5,
skip_file_check=False, prompt_cleanup=True, deduplicate_loras=True) -> tuple[List[Tuple[AnyStr, float]], str]:
found_loras = []
prompt_without_loras = ''
cleaned_prompt = ''
for token in prompt.split(','):
matches = LORAS_PROMPT_PATTERN.findall(token)
if len(matches) == 0:
prompt_without_loras += token + ', '
continue
for match in matches:
lora_name = match[1] + '.safetensors'
if not skip_file_check:
lora_name = get_filname_by_stem(match[1], modules.config.lora_filenames_no_special)
if lora_name is not None:
found_loras.append((lora_name, float(match[2])))
token = token.replace(match[0], '')
prompt_without_loras += token + ', '
if prompt_without_loras != '':
cleaned_prompt = prompt_without_loras[:-2]
if prompt_cleanup:
cleaned_prompt = cleanup_prompt(prompt_without_loras)
new_loras = []
lora_names = [lora[0] for lora in loras]
for found_lora in found_loras:
if deduplicate_loras and (found_lora[0] in lora_names or found_lora in new_loras):
continue
new_loras.append(found_lora)
if len(new_loras) == 0:
return loras, cleaned_prompt
updated_loras = []
for lora in loras + new_loras:
if lora[0] != "None":
updated_loras.append(lora)
return updated_loras[:loras_limit], cleaned_prompt
def cleanup_prompt(prompt):
prompt = re.sub(' +', ' ', prompt)
prompt = re.sub(',+', ',', prompt)
cleaned_prompt = ''
for token in prompt.split(','):
token = token.strip()
if token == '':
continue
cleaned_prompt += token + ', '
return cleaned_prompt[:-2]
def apply_wildcards(wildcard_text, rng, i, read_wildcards_in_order) -> str:
for _ in range(modules.config.wildcards_max_bfs_depth):
placeholders = re.findall(r'__([\w-]+)__', wildcard_text)
if len(placeholders) == 0:
return wildcard_text
print(f'[Wildcards] processing: {wildcard_text}')
for placeholder in placeholders:
try:
matches = [x for x in modules.config.wildcard_filenames if os.path.splitext(os.path.basename(x))[0] == placeholder]
words = open(os.path.join(modules.config.path_wildcards, matches[0]), encoding='utf-8').read().splitlines()
words = [x for x in words if x != '']
assert len(words) > 0
if read_wildcards_in_order:
wildcard_text = wildcard_text.replace(f'__{placeholder}__', words[i % len(words)], 1)
else:
wildcard_text = wildcard_text.replace(f'__{placeholder}__', rng.choice(words), 1)
except:
print(f'[Wildcards] Warning: {placeholder}.txt missing or empty. '
f'Using "{placeholder}" as a normal word.')
wildcard_text = wildcard_text.replace(f'__{placeholder}__', placeholder)
print(f'[Wildcards] {wildcard_text}')
print(f'[Wildcards] BFS stack overflow. Current text: {wildcard_text}')
return wildcard_text
def get_image_size_info(image: np.ndarray, aspect_ratios: list) -> str:
try:
image = Image.fromarray(np.uint8(image))
width, height = image.size
ratio = round(width / height, 2)
gcd = math.gcd(width, height)
lcm_ratio = f'{width // gcd}:{height // gcd}'
size_info = f'Image Size: {width} x {height}, Ratio: {ratio}, {lcm_ratio}'
closest_ratio = min(aspect_ratios, key=lambda x: abs(ratio - float(x.split('*')[0]) / float(x.split('*')[1])))
recommended_width, recommended_height = map(int, closest_ratio.split('*'))
recommended_ratio = round(recommended_width / recommended_height, 2)
recommended_gcd = math.gcd(recommended_width, recommended_height)
recommended_lcm_ratio = f'{recommended_width // recommended_gcd}:{recommended_height // recommended_gcd}'
size_info = f'{width} x {height}, {ratio}, {lcm_ratio}'
size_info += f'\n{recommended_width} x {recommended_height}, {recommended_ratio}, {recommended_lcm_ratio}'
return size_info
except Exception as e:
return f'Error reading image: {e}'

6
presets/.gitignore vendored Normal file
View File

@ -0,0 +1,6 @@
*.json
!anime.json
!default.json
!lcm.json
!realistic.json
!sai.json

23
presets/anime.json
View File

@ -1,46 +1,57 @@
{
"default_model": "animaPencilXL_v100.safetensors",
"default_model": "animaPencilXL_v310.safetensors",
"default_refiner": "None",
"default_refiner_switch": 0.5,
"default_loras": [
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
]
],
"default_cfg_scale": 7.0,
"default_cfg_scale": 6.0,
"default_sample_sharpness": 2.0,
"default_sampler": "dpmpp_2m_sde_gpu",
"default_scheduler": "karras",
"default_performance": "Speed",
"default_prompt": "1girl, ",
"default_prompt": "",
"default_prompt_negative": "",
"default_styles": [
"Fooocus V2",
"Fooocus Negative",
"Fooocus Semi Realistic",
"Fooocus Masterpiece"
],
"default_aspect_ratio": "896*1152",
"checkpoint_downloads": {
"animaPencilXL_v100.safetensors": "https://huggingface.co/lllyasviel/fav_models/resolve/main/fav/animaPencilXL_v100.safetensors"
"animaPencilXL_v310.safetensors": "https://huggingface.co/mashb1t/fav_models/resolve/main/fav/animaPencilXL_v310.safetensors"
},
"embeddings_downloads": {},
"lora_downloads": {},
"previous_default_models": []
"previous_default_models": [
"animaPencilXL_v300.safetensors",
"animaPencilXL_v260.safetensors",
"animaPencilXL_v210.safetensors",
"animaPencilXL_v200.safetensors",
"animaPencilXL_v100.safetensors"
]
}

5
presets/default.json
View File

@ -4,22 +4,27 @@
"default_refiner_switch": 0.5,
"default_loras": [
[
true,
"sd_xl_offset_example-lora_1.0.safetensors",
0.1
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
]

5
presets/lcm.json
View File

@ -4,22 +4,27 @@
"default_refiner_switch": 0.5,
"default_loras": [
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
]

57
presets/lightning.json Normal file
View File

@ -0,0 +1,57 @@
{
"default_model": "juggernautXL_v8Rundiffusion.safetensors",
"default_refiner": "None",
"default_refiner_switch": 0.5,
"default_loras": [
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
]
],
"default_cfg_scale": 4.0,
"default_sample_sharpness": 2.0,
"default_sampler": "dpmpp_2m_sde_gpu",
"default_scheduler": "karras",
"default_performance": "Lightning",
"default_prompt": "",
"default_prompt_negative": "",
"default_styles": [
"Fooocus V2",
"Fooocus Enhance",
"Fooocus Sharp"
],
"default_aspect_ratio": "1152*896",
"checkpoint_downloads": {
"juggernautXL_v8Rundiffusion.safetensors": "https://huggingface.co/lllyasviel/fav_models/resolve/main/fav/juggernautXL_v8Rundiffusion.safetensors"
},
"embeddings_downloads": {},
"lora_downloads": {},
"previous_default_models": [
"juggernautXL_version8Rundiffusion.safetensors",
"juggernautXL_version7Rundiffusion.safetensors",
"juggernautXL_v7Rundiffusion.safetensors",
"juggernautXL_version6Rundiffusion.safetensors",
"juggernautXL_v6Rundiffusion.safetensors"
]
}

7
presets/realistic.json
View File

@ -1,25 +1,30 @@
{
"default_model": "realisticStockPhoto_v20.safetensors",
"default_refiner": "",
"default_refiner": "None",
"default_refiner_switch": 0.5,
"default_loras": [
[
true,
"SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors",
0.25
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
]

5
presets/sai.json
View File

@ -4,22 +4,27 @@
"default_refiner_switch": 0.75,
"default_loras": [
[
true,
"sd_xl_offset_example-lora_1.0.safetensors",
0.5
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
],
[
true,
"None",
1.0
]

34
readme.md
View File

@ -84,6 +84,10 @@ The first time you launch the software, it will automatically download models:
After Fooocus 2.1.60, you will also have `run_anime.bat` and `run_realistic.bat`. They are different model presets (and require different models, but they will be automatically downloaded). [Check here for more details](https://github.com/lllyasviel/Fooocus/discussions/679).
After Fooocus 2.3.0 you can also switch presets directly in the browser. Keep in mind to add these arguments if you want to change the default behavior:
* Use `--disable-preset-selection` to disable preset selection in the browser.
* Use `--always-download-new-model` to download missing models on preset switch. Default is fallback to `previous_default_models` defined in the corresponding preset, also see terminal output.
![image](https://github.com/lllyasviel/Fooocus/assets/19834515/d386f817-4bd7-490c-ad89-c1e228c23447)
If you already have these files, you can copy them to the above locations to speed up installation.
@ -115,17 +119,21 @@ See also the common problems and troubleshoots [here](troubleshoot.md).
### Colab
(Last tested - 2023 Dec 12)
(Last tested - 2024 Mar 18 by [mashb1t](https://github.com/mashb1t))
| Colab | Info
| --- | --- |
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lllyasviel/Fooocus/blob/main/fooocus_colab.ipynb) | Fooocus Official
In Colab, you can modify the last line to `!python entry_with_update.py --share` or `!python entry_with_update.py --preset anime --share` or `!python entry_with_update.py --preset realistic --share` for Fooocus Default/Anime/Realistic Edition.
In Colab, you can modify the last line to `!python entry_with_update.py --share --always-high-vram` or `!python entry_with_update.py --share --always-high-vram --preset anime` or `!python entry_with_update.py --share --always-high-vram --preset realistic` for Fooocus Default/Anime/Realistic Edition.
You can also change the preset in the UI. Please be aware that this may lead to timeouts after 60 seconds. If this is the case, please wait until the download has finished, change the preset to initial and back to the one you've selected or reload the page.
Note that this Colab will disable refiner by default because Colab free's resources are relatively limited (and some "big" features like image prompt may cause free-tier Colab to disconnect). We make sure that basic text-to-image is always working on free-tier Colab.
Thanks to [camenduru](https://github.com/camenduru)!
Using `--always-high-vram` shifts resource allocation from RAM to VRAM and achieves the overall best balance between performance, flexibility and stability on the default T4 instance. Please find more information [here](https://github.com/lllyasviel/Fooocus/pull/1710#issuecomment-1989185346).
Thanks to [camenduru](https://github.com/camenduru) for the template!
### Linux (Using Anaconda)
@ -202,7 +210,7 @@ AMD is not intensively tested, however. The AMD support is in beta.
Use `python entry_with_update.py --preset anime` or `python entry_with_update.py --preset realistic` for Fooocus Anime/Realistic Edition.
### Windows(AMD GPUs)
### Windows (AMD GPUs)
Note that the [minimal requirement](#minimal-requirement) for different platforms is different.
@ -237,6 +245,10 @@ You can install Fooocus on Apple Mac silicon (M1 or M2) with macOS 'Catalina' or
Use `python entry_with_update.py --preset anime` or `python entry_with_update.py --preset realistic` for Fooocus Anime/Realistic Edition.
### Docker
See [docker.md](docker.md)
### Download Previous Version
See the guidelines [here](https://github.com/lllyasviel/Fooocus/discussions/1405).
@ -281,14 +293,21 @@ Given different goals, the default models and configs of Fooocus are different:
Note that the download is **automatic** - you do not need to do anything if the internet connection is okay. However, you can download them manually if you (or move them from somewhere else) have your own preparation.
## UI Access and Authentication
In addition to running on localhost, Fooocus can also expose its UI in two ways:
* Local UI listener: use `--listen` (specify port e.g. with `--port 8888`).
* API access: use `--share` (registers an endpoint at `.gradio.live`).
In both ways the access is unauthenticated by default. You can add basic authentication by creating a file called `auth.json` in the main directory, which contains a list of JSON objects with the keys `user` and `pass` (see example in [auth-example.json](./auth-example.json)).
## List of "Hidden" Tricks
<a name="tech_list"></a>
The below things are already inside the software, and **users do not need to do anything about these**.
1. GPT2-based [prompt expansion as a dynamic style "Fooocus V2".](https://github.com/lllyasviel/Fooocus/discussions/117#raw) (similar to Midjourney's hidden pre-processsing and "raw" mode, or the LeonardoAI's Prompt Magic).
1. GPT2-based [prompt expansion as a dynamic style "Fooocus V2".](https://github.com/lllyasviel/Fooocus/discussions/117#raw) (similar to Midjourney's hidden pre-processing and "raw" mode, or the LeonardoAI's Prompt Magic).
2. Native refiner swap inside one single k-sampler. The advantage is that the refiner model can now reuse the base model's momentum (or ODE's history parameters) collected from k-sampling to achieve more coherent sampling. In Automatic1111's high-res fix and ComfyUI's node system, the base model and refiner use two independent k-samplers, which means the momentum is largely wasted, and the sampling continuity is broken. Fooocus uses its own advanced k-diffusion sampling that ensures seamless, native, and continuous swap in a refiner setup. (Update Aug 13: Actually, I discussed this with Automatic1111 several days ago, and it seems that the “native refiner swap inside one single k-sampler” is [merged]( https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12371) into the dev branch of webui. Great!)
3. Negative ADM guidance. Because the highest resolution level of XL Base does not have cross attentions, the positive and negative signals for XL's highest resolution level cannot receive enough contrasts during the CFG sampling, causing the results to look a bit plastic or overly smooth in certain cases. Fortunately, since the XL's highest resolution level is still conditioned on image aspect ratios (ADM), we can modify the adm on the positive/negative side to compensate for the lack of CFG contrast in the highest resolution level. (Update Aug 16, the IOS App [Drawing Things](https://apps.apple.com/us/app/draw-things-ai-generation/id6444050820) will support Negative ADM Guidance. Great!)
3. Negative ADM guidance. Because the highest resolution level of XL Base does not have cross attentions, the positive and negative signals for XL's highest resolution level cannot receive enough contrasts during the CFG sampling, causing the results to look a bit plastic or overly smooth in certain cases. Fortunately, since the XL's highest resolution level is still conditioned on image aspect ratios (ADM), we can modify the adm on the positive/negative side to compensate for the lack of CFG contrast in the highest resolution level. (Update Aug 16, the IOS App [Draw Things](https://apps.apple.com/us/app/draw-things-ai-generation/id6444050820) will support Negative ADM Guidance. Great!)
4. We implemented a carefully tuned variation of Section 5.1 of ["Improving Sample Quality of Diffusion Models Using Self-Attention Guidance"](https://arxiv.org/pdf/2210.00939.pdf). The weight is set to very low, but this is Fooocus's final guarantee to make sure that the XL will never yield an overly smooth or plastic appearance (examples [here](https://github.com/lllyasviel/Fooocus/discussions/117#sharpness)). This can almost eliminate all cases for which XL still occasionally produces overly smooth results, even with negative ADM guidance. (Update 2023 Aug 18, the Gaussian kernel of SAG is changed to an anisotropic kernel for better structure preservation and fewer artifacts.)
5. We modified the style templates a bit and added the "cinematic-default".
6. We tested the "sd_xl_offset_example-lora_1.0.safetensors" and it seems that when the lora weight is below 0.5, the results are always better than XL without lora.
@ -349,6 +368,7 @@ A safer way is just to try "run_anime.bat" or "run_realistic.bat" - they should
entry_with_update.py [-h] [--listen [IP]] [--port PORT]
[--disable-header-check [ORIGIN]]
[--web-upload-size WEB_UPLOAD_SIZE]
[--hf-mirror HF_MIRROR]
[--external-working-path PATH [PATH ...]]
[--output-path OUTPUT_PATH] [--temp-path TEMP_PATH]
[--cache-path CACHE_PATH] [--in-browser]
@ -363,7 +383,7 @@ entry_with_update.py [-h] [--listen [IP]] [--port PORT]
[--attention-split | --attention-quad | --attention-pytorch]
[--disable-xformers]
[--always-gpu | --always-high-vram | --always-normal-vram |
--always-low-vram | --always-no-vram | --always-cpu]
--always-low-vram | --always-no-vram | --always-cpu [CPU_NUM_THREADS]]
[--always-offload-from-vram] [--disable-server-log]
[--debug-mode] [--is-windows-embedded-python]
[--disable-server-info] [--share] [--preset PRESET]

2
requirements_docker.txt Normal file
View File

@ -0,0 +1,2 @@
torch==2.1.0
torchvision==0.16.0

BIN
sdxl_styles/samples/fooocus_semi_realistic.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.4 KiB

BIN
sdxl_styles/samples/random_style.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 KiB

4
sdxl_styles/sdxl_styles_fooocus.json
View File

@ -3,6 +3,10 @@
"name": "Fooocus Enhance",
"negative_prompt": "(worst quality, low quality, normal quality, lowres, low details, oversaturated, undersaturated, overexposed, underexposed, grayscale, bw, bad photo, bad photography, bad art:1.4), (watermark, signature, text font, username, error, logo, words, letters, digits, autograph, trademark, name:1.2), (blur, blurry, grainy), morbid, ugly, asymmetrical, mutated malformed, mutilated, poorly lit, bad shadow, draft, cropped, out of frame, cut off, censored, jpeg artifacts, out of focus, glitch, duplicate, (airbrushed, cartoon, anime, semi-realistic, cgi, render, blender, digital art, manga, amateur:1.3), (3D ,3D Game, 3D Game Scene, 3D Character:1.1), (bad hands, bad anatomy, bad body, bad face, bad teeth, bad arms, bad legs, deformities:1.3)"
},
{
"name": "Fooocus Semi Realistic",
"negative_prompt": "(worst quality, low quality, normal quality, lowres, low details, oversaturated, undersaturated, overexposed, underexposed, bad photo, bad photography, bad art:1.4), (watermark, signature, text font, username, error, logo, words, letters, digits, autograph, trademark, name:1.2), (blur, blurry, grainy), morbid, ugly, asymmetrical, mutated malformed, mutilated, poorly lit, bad shadow, draft, cropped, out of frame, cut off, censored, jpeg artifacts, out of focus, glitch, duplicate, (bad hands, bad anatomy, bad body, bad face, bad teeth, bad arms, bad legs, deformities:1.3)"
},
{
"name": "Fooocus Sharp",
"prompt": "cinematic still {prompt} . emotional, harmonious, vignette, 4k epic detailed, shot on kodak, 35mm photo, sharp focus, high budget, cinemascope, moody, epic, gorgeous, film grain, grainy",

3
shared.py
View File

@ -1,2 +1 @@
gradio_root = None
last_stop = None
gradio_root = None

4
tests/__init__.py Normal file
View File

@ -0,0 +1,4 @@
import sys
import pathlib
sys.path.append(pathlib.Path(f'{__file__}/../modules').parent.resolve())

81
tests/test_utils.py Normal file
View File

@ -0,0 +1,81 @@
import unittest
from modules import util
class TestUtils(unittest.TestCase):
def test_can_parse_tokens_with_lora(self):
test_cases = [
{
"input": ("some prompt, very cool, <lora:hey-lora:0.4>, cool <lora:you-lora:0.2>", [], 5, True),
"output": (
[('hey-lora.safetensors', 0.4), ('you-lora.safetensors', 0.2)], 'some prompt, very cool, cool'),
},
# Test can not exceed limit
{
"input": ("some prompt, very cool, <lora:hey-lora:0.4>, cool <lora:you-lora:0.2>", [], 1, True),
"output": (
[('hey-lora.safetensors', 0.4)],
'some prompt, very cool, cool'
),
},
# test Loras from UI take precedence over prompt
{
"input": (
"some prompt, very cool, <lora:l1:0.4>, <lora:l2:-0.2>, <lora:l3:0.3>, <lora:l4:0.5>, <lora:l6:0.24>, <lora:l7:0.1>",
[("hey-lora.safetensors", 0.4)],
5,
True
),
"output": (
[
('hey-lora.safetensors', 0.4),
('l1.safetensors', 0.4),
('l2.safetensors', -0.2),
('l3.safetensors', 0.3),
('l4.safetensors', 0.5)
],
'some prompt, very cool'
)
},
# test correct matching even if there is no space separating loras in the same token
{
"input": ("some prompt, very cool, <lora:hey-lora:0.4><lora:you-lora:0.2>", [], 3, True),
"output": (
[
('hey-lora.safetensors', 0.4),
('you-lora.safetensors', 0.2)
],
'some prompt, very cool'
),
},
# test deduplication, also selected loras are never overridden with loras in prompt
{
"input": (
"some prompt, very cool, <lora:hey-lora:0.4><lora:hey-lora:0.4><lora:you-lora:0.2>",
[('you-lora.safetensors', 0.3)],
3,
True
),
"output": (
[
('you-lora.safetensors', 0.3),
('hey-lora.safetensors', 0.4)
],
'some prompt, very cool'
),
},
{
"input": ("<lora:foo:1..2>, <lora:bar:.>, <test:1.0>, <lora:baz:+> and <lora:quux:>", [], 6, True),
"output": (
[],
'<lora:foo:1..2>, <lora:bar:.>, <test:1.0>, <lora:baz:+> and <lora:quux:>'
)
}
]
for test in test_cases:
prompt, loras, loras_limit, skip_file_check = test["input"]
expected = test["output"]
actual = util.parse_lora_references_from_prompt(prompt, loras, loras_limit=loras_limit, skip_file_check=skip_file_check)
self.assertEqual(expected, actual)

57
update_log.md
View File

@ -1,3 +1,60 @@
# [2.4.0](https://github.com/lllyasviel/Fooocus/releases/tag/v2.4.0)
* Change settings tab elements to be more compact
* Add clip skip slider
* Add select for custom VAE
* Add new style "Random Style"
* Update default anime model to animaPencilXL_v310
* Add button to reconnect the UI after Fooocus crashed without having to configure everything again (no page reload required)
* Add performance "hyper-sd" (based on [Hyper-SDXL 4 step LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-4steps-lora.safetensors))
* Add [AlignYourSteps](https://research.nvidia.com/labs/toronto-ai/AlignYourSteps/) scheduler by Nvidia, see
* Add [TCD](https://github.com/jabir-zheng/TCD) sampler and scheduler (based on sgm_uniform)
* Add NSFW image censoring (disables intermediate image preview while generating). Set config value `default_black_out_nsfw` to True to always enable.
* Add argument `--enable-describe-uov-image` to automatically describe uploaded images for upscaling
* Add inline lora prompt references with subfolder support, example prompt: `colorful bird <lora:toucan:1.2>`
* Add size and aspect ratio recommendation on image describe
* Add inpaint brush color picker, helpful when image and mask brush have the same color
* Add automated Docker image build using Github Actions on each release.
* Add full raw prompts to history logs
* Change code ownership from @lllyasviel to @mashb1t for automated issue / MR notification
# [2.3.1](https://github.com/lllyasviel/Fooocus/releases/tag/2.3.1)
* Remove positive prompt from anime prefix to not reset prompt after switching presets
* Fix image number being reset to 1 when switching preset, now doesn't reset anymore
* Fix outpainting dimension calculation when extending left/right
* Fix LoRA compatibility for LoRAs in a1111 metadata scheme
# [2.3.0](https://github.com/lllyasviel/Fooocus/releases/tag/2.3.0)
* Add performance "lightning" (based on [SDXL-Lightning 4 step LoRA](https://huggingface.co/ByteDance/SDXL-Lightning/blob/main/sdxl_lightning_4step_lora.safetensors))
* Add preset selection to UI, disable with argument `--disable-preset-selection`. Use `--always-download-new-model` to download missing models on preset switch.
* Improve face swap consistency by switching later in the process to (synthetic) refiner
* Add temp path cleanup on startup
* Add support for wildcard subdirectories
* Add scrollable 2 column layout for styles for better structure
* Improve Colab resource needs for T4 instances (default), positively tested with all image prompt features
* Improve anime preset, now uses style `Fooocus Semi Realistic` instead of `Fooocus Negative` (less wet look images)
# [2.2.1](https://github.com/lllyasviel/Fooocus/releases/tag/2.2.1)
* Fix some small bugs (e.g. image grid, upscale fast 2x, LoRA weight width in Firefox)
* Allow prompt weights in array syntax
* Add steps override and metadata scheme to history log
# [2.2.0](https://github.com/lllyasviel/Fooocus/releases/tag/2.2.0)
* Isolate every image generation to truly allow multi-user usage
* Add array support, changes the main prompt when increasing the image number. Syntax: `[[red, green, blue]] flower`
* Add optional metadata to images, allowing you to regenerate and modify them later with the same parameters
* Now supports native PNG, JPG and WEBP image generation
* Add Docker support
# [2.1.865](https://github.com/lllyasviel/Fooocus/releases/tag/2.1.865)
* Various bugfixes
* Add authentication to --listen
# 2.1.864
* New model list. See also discussions.

366
webui.py
View File

@ -11,28 +11,35 @@ import modules.async_worker as worker
import modules.constants as constants
import modules.flags as flags
import modules.gradio_hijack as grh
import modules.advanced_parameters as advanced_parameters
import modules.style_sorter as style_sorter
import modules.meta_parser
import args_manager
import copy
import launch
from modules.sdxl_styles import legal_style_names
from modules.private_logger import get_current_html_path
from modules.ui_gradio_extensions import reload_javascript
from modules.auth import auth_enabled, check_auth
from modules.util import is_json
def get_task(*args):
args = list(args)
args.pop(0)
def generate_clicked(*args):
return worker.AsyncTask(args=args)
def generate_clicked(task: worker.AsyncTask):
import ldm_patched.modules.model_management as model_management
with model_management.interrupt_processing_mutex:
model_management.interrupt_processing = False
# outputs=[progress_html, progress_window, progress_gallery, gallery]
if len(task.args) == 0:
return
execution_start_time = time.perf_counter()
task = worker.AsyncTask(args=list(args))
finished = False
yield gr.update(visible=True, value=modules.html.make_progress_html(1, 'Waiting for task to start ...')), \
@ -71,6 +78,12 @@ def generate_clicked(*args):
gr.update(visible=True, value=product)
finished = True
# delete Fooocus temp images, only keep gradio temp images
if args_manager.args.disable_image_log:
for filepath in product:
if isinstance(filepath, str) and os.path.exists(filepath):
os.remove(filepath)
execution_time = time.perf_counter() - execution_start_time
print(f'Total time: {execution_time:.2f} seconds')
return
@ -83,11 +96,10 @@ title = f'Fooocus {fooocus_version.version}'
if isinstance(args_manager.args.preset, str):
title += ' ' + args_manager.args.preset
shared.gradio_root = gr.Blocks(
title=title,
css=modules.html.css).queue()
shared.gradio_root = gr.Blocks(title=title).queue()
with shared.gradio_root:
currentTask = gr.State(worker.AsyncTask(args=[]))
with gr.Row():
with gr.Column(scale=2):
with gr.Row():
@ -111,25 +123,27 @@ with shared.gradio_root:
with gr.Column(scale=3, min_width=0):
generate_button = gr.Button(label="Generate", value="Generate", elem_classes='type_row', elem_id='generate_button', visible=True)
reset_button = gr.Button(label="Reconnect", value="Reconnect", elem_classes='type_row', elem_id='reset_button', visible=False)
load_parameter_button = gr.Button(label="Load Parameters", value="Load Parameters", elem_classes='type_row', elem_id='load_parameter_button', visible=False)
skip_button = gr.Button(label="Skip", value="Skip", elem_classes='type_row_half', visible=False)
skip_button = gr.Button(label="Skip", value="Skip", elem_classes='type_row_half', elem_id='skip_button', visible=False)
stop_button = gr.Button(label="Stop", value="Stop", elem_classes='type_row_half', elem_id='stop_button', visible=False)
def stop_clicked():
def stop_clicked(currentTask):
import ldm_patched.modules.model_management as model_management
shared.last_stop = 'stop'
model_management.interrupt_current_processing()
return [gr.update(interactive=False)] * 2
currentTask.last_stop = 'stop'
if (currentTask.processing):
model_management.interrupt_current_processing()
return currentTask
def skip_clicked():
def skip_clicked(currentTask):
import ldm_patched.modules.model_management as model_management
shared.last_stop = 'skip'
model_management.interrupt_current_processing()
return
currentTask.last_stop = 'skip'
if (currentTask.processing):
model_management.interrupt_current_processing()
return currentTask
stop_button.click(stop_clicked, outputs=[skip_button, stop_button],
queue=False, show_progress=False, _js='cancelGenerateForever')
skip_button.click(skip_clicked, queue=False, show_progress=False)
stop_button.click(stop_clicked, inputs=currentTask, outputs=currentTask, queue=False, show_progress=False, _js='cancelGenerateForever')
skip_button.click(skip_clicked, inputs=currentTask, outputs=currentTask, queue=False, show_progress=False)
with gr.Row(elem_classes='advanced_check_row'):
input_image_checkbox = gr.Checkbox(label='Input Image', value=False, container=False, elem_classes='min_check')
advanced_checkbox = gr.Checkbox(label='Advanced', value=modules.config.default_advanced_checkbox, container=False, elem_classes='min_check')
@ -138,7 +152,7 @@ with shared.gradio_root:
with gr.TabItem(label='Upscale or Variation') as uov_tab:
with gr.Row():
with gr.Column():
uov_input_image = grh.Image(label='Drag above image to here', source='upload', type='numpy')
uov_input_image = grh.Image(label='Image', source='upload', type='numpy', show_label=False)
with gr.Column():
uov_method = gr.Radio(label='Upscale or Variation:', choices=flags.uov_list, value=flags.disabled)
gr.HTML('<a href="https://github.com/lllyasviel/Fooocus/discussions/390" target="_blank">\U0001F4D4 Document</a>')
@ -150,7 +164,7 @@ with shared.gradio_root:
ip_weights = []
ip_ctrls = []
ip_ad_cols = []
for _ in range(4):
for _ in range(flags.controlnet_image_count):
with gr.Column():
ip_image = grh.Image(label='Image', source='upload', type='numpy', show_label=False, height=300)
ip_images.append(ip_image)
@ -187,7 +201,7 @@ with shared.gradio_root:
queue=False, show_progress=False)
with gr.TabItem(label='Inpaint or Outpaint') as inpaint_tab:
with gr.Row():
inpaint_input_image = grh.Image(label='Drag inpaint or outpaint image to here', source='upload', type='numpy', tool='sketch', height=500, brush_color="#FFFFFF", elem_id='inpaint_canvas')
inpaint_input_image = grh.Image(label='Image', source='upload', type='numpy', tool='sketch', height=500, brush_color="#FFFFFF", elem_id='inpaint_canvas', show_label=False)
inpaint_mask_image = grh.Image(label='Mask Upload', source='upload', type='numpy', height=500, visible=False)
with gr.Row():
@ -200,14 +214,44 @@ with shared.gradio_root:
with gr.TabItem(label='Describe') as desc_tab:
with gr.Row():
with gr.Column():
desc_input_image = grh.Image(label='Drag any image to here', source='upload', type='numpy')
desc_input_image = grh.Image(label='Image', source='upload', type='numpy', show_label=False)
with gr.Column():
desc_method = gr.Radio(
label='Content Type',
choices=[flags.desc_type_photo, flags.desc_type_anime],
value=flags.desc_type_photo)
desc_btn = gr.Button(value='Describe this Image into Prompt')
desc_image_size = gr.Textbox(label='Image Size and Recommended Size', elem_id='desc_image_size', visible=False)
gr.HTML('<a href="https://github.com/lllyasviel/Fooocus/discussions/1363" target="_blank">\U0001F4D4 Document</a>')
def trigger_show_image_properties(image):
value = modules.util.get_image_size_info(image, modules.flags.sdxl_aspect_ratios)
return gr.update(value=value, visible=True)
desc_input_image.upload(trigger_show_image_properties, inputs=desc_input_image,
outputs=desc_image_size, show_progress=False, queue=False)
with gr.TabItem(label='Metadata') as metadata_tab:
with gr.Column():
metadata_input_image = grh.Image(label='For images created by Fooocus', source='upload', type='filepath')
metadata_json = gr.JSON(label='Metadata')
metadata_import_button = gr.Button(value='Apply Metadata')
def trigger_metadata_preview(filepath):
parameters, metadata_scheme = modules.meta_parser.read_info_from_image(filepath)
results = {}
if parameters is not None:
results['parameters'] = parameters
if isinstance(metadata_scheme, flags.MetadataScheme):
results['metadata_scheme'] = metadata_scheme.value
return results
metadata_input_image.upload(trigger_metadata_preview, inputs=metadata_input_image,
outputs=metadata_json, queue=False, show_progress=True)
switch_js = "(x) => {if(x){viewer_to_bottom(100);viewer_to_bottom(500);}else{viewer_to_top();} return x;}"
down_js = "() => {viewer_to_bottom();}"
@ -220,16 +264,35 @@ with shared.gradio_root:
inpaint_tab.select(lambda: 'inpaint', outputs=current_tab, queue=False, _js=down_js, show_progress=False)
ip_tab.select(lambda: 'ip', outputs=current_tab, queue=False, _js=down_js, show_progress=False)
desc_tab.select(lambda: 'desc', outputs=current_tab, queue=False, _js=down_js, show_progress=False)
metadata_tab.select(lambda: 'metadata', outputs=current_tab, queue=False, _js=down_js, show_progress=False)
with gr.Column(scale=1, visible=modules.config.default_advanced_checkbox) as advanced_column:
with gr.Tab(label='Setting'):
if not args_manager.args.disable_preset_selection:
preset_selection = gr.Dropdown(label='Preset',
choices=modules.config.available_presets,
value=args_manager.args.preset if args_manager.args.preset else "initial",
interactive=True)
performance_selection = gr.Radio(label='Performance',
choices=modules.flags.performance_selections,
value=modules.config.default_performance)
aspect_ratios_selection = gr.Radio(label='Aspect Ratios', choices=modules.config.available_aspect_ratios,
value=modules.config.default_aspect_ratio, info='width × height',
elem_classes='aspect_ratios')
choices=flags.Performance.list(),
value=modules.config.default_performance,
elem_classes=['performance_selection'])
with gr.Accordion(label='Aspect Ratios', open=False, elem_id='aspect_ratios_accordion') as aspect_ratios_accordion:
aspect_ratios_selection = gr.Radio(label='Aspect Ratios', show_label=False,
choices=modules.config.available_aspect_ratios_labels,
value=modules.config.default_aspect_ratio,
info='width × height',
elem_classes='aspect_ratios')
aspect_ratios_selection.change(lambda x: None, inputs=aspect_ratios_selection, queue=False, show_progress=False, _js='(x)=>{refresh_aspect_ratios_label(x);}')
shared.gradio_root.load(lambda x: None, inputs=aspect_ratios_selection, queue=False, show_progress=False, _js='(x)=>{refresh_aspect_ratios_label(x);}')
image_number = gr.Slider(label='Image Number', minimum=1, maximum=modules.config.default_max_image_number, step=1, value=modules.config.default_image_number)
output_format = gr.Radio(label='Output Format',
choices=flags.OutputFormat.list(),
value=modules.config.default_output_format)
negative_prompt = gr.Textbox(label='Negative Prompt', show_label=True, placeholder="Type prompt here.",
info='Describing what you do not want to see.', lines=2,
elem_id='negative_prompt',
@ -255,10 +318,16 @@ with shared.gradio_root:
seed_random.change(random_checked, inputs=[seed_random], outputs=[image_seed],
queue=False, show_progress=False)
if not args_manager.args.disable_image_log:
gr.HTML(f'<a href="file={get_current_html_path()}" target="_blank">\U0001F4DA History Log</a>')
def update_history_link():
if args_manager.args.disable_image_log:
return gr.update(value='')
return gr.update(value=f'<a href="file={get_current_html_path(output_format)}" target="_blank">\U0001F4DA History Log</a>')
with gr.Tab(label='Style'):
history_link = gr.HTML()
shared.gradio_root.load(update_history_link, outputs=history_link, queue=False, show_progress=False)
with gr.Tab(label='Style', elem_classes=['style_selections_tab']):
style_sorter.try_load_sorted_styles(
style_names=legal_style_names,
default_selected=modules.config.default_styles)
@ -311,16 +380,20 @@ with shared.gradio_root:
with gr.Group():
lora_ctrls = []
for i, (n, v) in enumerate(modules.config.default_loras):
for i, (enabled, filename, weight) in enumerate(modules.config.default_loras):
with gr.Row():
lora_enabled = gr.Checkbox(label='Enable', value=enabled,
elem_classes=['lora_enable', 'min_check'], scale=1)
lora_model = gr.Dropdown(label=f'LoRA {i + 1}',
choices=['None'] + modules.config.lora_filenames, value=n)
lora_weight = gr.Slider(label='Weight', minimum=-2, maximum=2, step=0.01, value=v,
elem_classes='lora_weight')
lora_ctrls += [lora_model, lora_weight]
choices=['None'] + modules.config.lora_filenames, value=filename,
elem_classes='lora_model', scale=5)
lora_weight = gr.Slider(label='Weight', minimum=modules.config.default_loras_min_weight,
maximum=modules.config.default_loras_max_weight, step=0.01, value=weight,
elem_classes='lora_weight', scale=5)
lora_ctrls += [lora_enabled, lora_model, lora_weight]
with gr.Row():
model_refresh = gr.Button(label='Refresh', value='\U0001f504 Refresh All Files', variant='secondary', elem_classes='refresh_button')
refresh_files = gr.Button(label='Refresh', value='\U0001f504 Refresh All Files', variant='secondary', elem_classes='refresh_button')
with gr.Tab(label='Advanced'):
guidance_scale = gr.Slider(label='Guidance Scale', minimum=1.0, maximum=30.0, step=0.01,
value=modules.config.default_cfg_scale,
@ -341,17 +414,22 @@ with shared.gradio_root:
step=0.001, value=0.3,
info='When to end the guidance from positive/negative ADM. ')
refiner_swap_method = gr.Dropdown(label='Refiner swap method', value='joint',
refiner_swap_method = gr.Dropdown(label='Refiner swap method', value=flags.refiner_swap_method,
choices=['joint', 'separate', 'vae'])
adaptive_cfg = gr.Slider(label='CFG Mimicking from TSNR', minimum=1.0, maximum=30.0, step=0.01,
value=modules.config.default_cfg_tsnr,
info='Enabling Fooocus\'s implementation of CFG mimicking for TSNR '
'(effective when real CFG > mimicked CFG).')
clip_skip = gr.Slider(label='CLIP Skip', minimum=1, maximum=flags.clip_skip_max, step=1,
value=modules.config.default_clip_skip,
info='Bypass CLIP layers to avoid overfitting (use 1 to not skip any layers, 2 is recommended).')
sampler_name = gr.Dropdown(label='Sampler', choices=flags.sampler_list,
value=modules.config.default_sampler)
scheduler_name = gr.Dropdown(label='Scheduler', choices=flags.scheduler_list,
value=modules.config.default_scheduler)
vae_name = gr.Dropdown(label='VAE', choices=[modules.flags.default_vae] + modules.config.vae_filenames,
value=modules.config.default_vae, show_label=True)
generate_image_grid = gr.Checkbox(label='Generate Image Grid for Each Batch',
info='(Experimental) This may cause performance problems on some computers and certain internet conditions.',
@ -379,8 +457,36 @@ with shared.gradio_root:
overwrite_upscale_strength = gr.Slider(label='Forced Overwrite of Denoising Strength of "Upscale"',
minimum=-1, maximum=1.0, step=0.001, value=-1,
info='Set as negative number to disable. For developer debugging.')
disable_preview = gr.Checkbox(label='Disable Preview', value=False,
disable_preview = gr.Checkbox(label='Disable Preview', value=modules.config.default_black_out_nsfw,
interactive=not modules.config.default_black_out_nsfw,
info='Disable preview during generation.')
disable_intermediate_results = gr.Checkbox(label='Disable Intermediate Results',
value=modules.config.default_performance == flags.Performance.EXTREME_SPEED.value,
interactive=modules.config.default_performance != flags.Performance.EXTREME_SPEED.value,
info='Disable intermediate results during generation, only show final gallery.')
disable_seed_increment = gr.Checkbox(label='Disable seed increment',
info='Disable automatic seed increment when image number is > 1.',
value=False)
read_wildcards_in_order = gr.Checkbox(label="Read wildcards in order", value=False)
black_out_nsfw = gr.Checkbox(label='Black Out NSFW',
value=modules.config.default_black_out_nsfw,
interactive=not modules.config.default_black_out_nsfw,
info='Use black image if NSFW is detected.')
black_out_nsfw.change(lambda x: gr.update(value=x, interactive=not x),
inputs=black_out_nsfw, outputs=disable_preview, queue=False,
show_progress=False)
if not args_manager.args.disable_metadata:
save_metadata_to_images = gr.Checkbox(label='Save Metadata to Images', value=modules.config.default_save_metadata_to_images,
info='Adds parameters to generated images allowing manual regeneration.')
metadata_scheme = gr.Radio(label='Metadata Scheme', choices=flags.metadata_scheme, value=modules.config.default_metadata_scheme,
info='Image Prompt parameters are not included. Use png and a1111 for compatibility with Civitai.',
visible=modules.config.default_save_metadata_to_images)
save_metadata_to_images.change(lambda x: gr.update(visible=x), inputs=[save_metadata_to_images], outputs=[metadata_scheme],
queue=False, show_progress=False)
with gr.Tab(label='Control'):
debugging_cn_preprocessor = gr.Checkbox(label='Debug Preprocessors', value=False,
@ -429,14 +535,21 @@ with shared.gradio_root:
'(default is 0, always process before any mask invert)')
inpaint_mask_upload_checkbox = gr.Checkbox(label='Enable Mask Upload', value=False)
invert_mask_checkbox = gr.Checkbox(label='Invert Mask', value=False)
inpaint_mask_color = gr.ColorPicker(label='Inpaint brush color', value='#FFFFFF', elem_id='inpaint_brush_color')
inpaint_ctrls = [debugging_inpaint_preprocessor, inpaint_disable_initial_latent, inpaint_engine,
inpaint_strength, inpaint_respective_field,
inpaint_mask_upload_checkbox, invert_mask_checkbox, inpaint_erode_or_dilate]
inpaint_mask_upload_checkbox.change(lambda x: gr.update(visible=x),
inputs=inpaint_mask_upload_checkbox,
outputs=inpaint_mask_image, queue=False, show_progress=False)
inputs=inpaint_mask_upload_checkbox,
outputs=inpaint_mask_image, queue=False,
show_progress=False)
inpaint_mask_color.change(lambda x: gr.update(brush_color=x), inputs=inpaint_mask_color,
outputs=inpaint_input_image,
queue=False, show_progress=False)
with gr.Tab(label='FreeU'):
freeu_enabled = gr.Checkbox(label='Enabled', value=False)
@ -446,42 +559,73 @@ with shared.gradio_root:
freeu_s2 = gr.Slider(label='S2', minimum=0, maximum=4, step=0.01, value=0.95)
freeu_ctrls = [freeu_enabled, freeu_b1, freeu_b2, freeu_s1, freeu_s2]
adps = [disable_preview, adm_scaler_positive, adm_scaler_negative, adm_scaler_end, adaptive_cfg, sampler_name,
scheduler_name, generate_image_grid, overwrite_step, overwrite_switch, overwrite_width, overwrite_height,
overwrite_vary_strength, overwrite_upscale_strength,
mixing_image_prompt_and_vary_upscale, mixing_image_prompt_and_inpaint,
debugging_cn_preprocessor, skipping_cn_preprocessor, controlnet_softness,
canny_low_threshold, canny_high_threshold, refiner_swap_method]
adps += freeu_ctrls
adps += inpaint_ctrls
def dev_mode_checked(r):
return gr.update(visible=r)
dev_mode.change(dev_mode_checked, inputs=[dev_mode], outputs=[dev_tools],
queue=False, show_progress=False)
def model_refresh_clicked():
modules.config.update_all_model_names()
results = []
results += [gr.update(choices=modules.config.model_filenames), gr.update(choices=['None'] + modules.config.model_filenames)]
for i in range(5):
results += [gr.update(choices=['None'] + modules.config.lora_filenames), gr.update()]
def refresh_files_clicked():
modules.config.update_files()
results = [gr.update(choices=modules.config.model_filenames)]
results += [gr.update(choices=['None'] + modules.config.model_filenames)]
results += [gr.update(choices=['None'] + modules.config.vae_filenames)]
if not args_manager.args.disable_preset_selection:
results += [gr.update(choices=modules.config.available_presets)]
for i in range(modules.config.default_max_lora_number):
results += [gr.update(interactive=True),
gr.update(choices=['None'] + modules.config.lora_filenames), gr.update()]
return results
model_refresh.click(model_refresh_clicked, [], [base_model, refiner_model] + lora_ctrls,
refresh_files_output = [base_model, refiner_model, vae_name]
if not args_manager.args.disable_preset_selection:
refresh_files_output += [preset_selection]
refresh_files.click(refresh_files_clicked, [], refresh_files_output + lora_ctrls,
queue=False, show_progress=False)
performance_selection.change(lambda x: [gr.update(interactive=x != 'Extreme Speed')] * 11 +
[gr.update(visible=x != 'Extreme Speed')] * 1,
state_is_generating = gr.State(False)
load_data_outputs = [advanced_checkbox, image_number, prompt, negative_prompt, style_selections,
performance_selection, overwrite_step, overwrite_switch, aspect_ratios_selection,
overwrite_width, overwrite_height, guidance_scale, sharpness, adm_scaler_positive,
adm_scaler_negative, adm_scaler_end, refiner_swap_method, adaptive_cfg, clip_skip,
base_model, refiner_model, refiner_switch, sampler_name, scheduler_name, vae_name,
seed_random, image_seed, generate_button, load_parameter_button] + freeu_ctrls + lora_ctrls
if not args_manager.args.disable_preset_selection:
def preset_selection_change(preset, is_generating):
preset_content = modules.config.try_get_preset_content(preset) if preset != 'initial' else {}
preset_prepared = modules.meta_parser.parse_meta_from_preset(preset_content)
default_model = preset_prepared.get('base_model')
previous_default_models = preset_prepared.get('previous_default_models', [])
checkpoint_downloads = preset_prepared.get('checkpoint_downloads', {})
embeddings_downloads = preset_prepared.get('embeddings_downloads', {})
lora_downloads = preset_prepared.get('lora_downloads', {})
preset_prepared['base_model'], preset_prepared['lora_downloads'] = launch.download_models(
default_model, previous_default_models, checkpoint_downloads, embeddings_downloads, lora_downloads)
if 'prompt' in preset_prepared and preset_prepared.get('prompt') == '':
del preset_prepared['prompt']
return modules.meta_parser.load_parameter_button_click(json.dumps(preset_prepared), is_generating)
preset_selection.change(preset_selection_change, inputs=[preset_selection, state_is_generating], outputs=load_data_outputs, queue=False, show_progress=True) \
.then(fn=style_sorter.sort_styles, inputs=style_selections, outputs=style_selections, queue=False, show_progress=False)
performance_selection.change(lambda x: [gr.update(interactive=not flags.Performance.has_restricted_features(x))] * 11 +
[gr.update(visible=not flags.Performance.has_restricted_features(x))] * 1 +
[gr.update(interactive=not flags.Performance.has_restricted_features(x), value=flags.Performance.has_restricted_features(x))] * 1,
inputs=performance_selection,
outputs=[
guidance_scale, sharpness, adm_scaler_end, adm_scaler_positive,
adm_scaler_negative, refiner_switch, refiner_model, sampler_name,
scheduler_name, adaptive_cfg, refiner_swap_method, negative_prompt
scheduler_name, adaptive_cfg, refiner_swap_method, negative_prompt, disable_intermediate_results
], queue=False, show_progress=False)
output_format.input(lambda x: gr.update(output_format=x), inputs=output_format)
advanced_checkbox.change(lambda x: gr.update(visible=x), advanced_checkbox, advanced_column,
queue=False, show_progress=False) \
.then(fn=lambda: None, _js='refresh_grid_delayed', queue=False, show_progress=False)
@ -519,29 +663,36 @@ with shared.gradio_root:
inpaint_strength, inpaint_respective_field
], show_progress=False, queue=False)
ctrls = [
ctrls = [currentTask, generate_image_grid]
ctrls += [
prompt, negative_prompt, style_selections,
performance_selection, aspect_ratios_selection, image_number, image_seed, sharpness, guidance_scale
performance_selection, aspect_ratios_selection, image_number, output_format, image_seed,
read_wildcards_in_order, sharpness, guidance_scale
]
ctrls += [base_model, refiner_model, refiner_switch] + lora_ctrls
ctrls += [input_image_checkbox, current_tab]
ctrls += [uov_method, uov_input_image]
ctrls += [outpaint_selections, inpaint_input_image, inpaint_additional_prompt, inpaint_mask_image]
ctrls += ip_ctrls
ctrls += [disable_preview, disable_intermediate_results, disable_seed_increment, black_out_nsfw]
ctrls += [adm_scaler_positive, adm_scaler_negative, adm_scaler_end, adaptive_cfg, clip_skip]
ctrls += [sampler_name, scheduler_name, vae_name]
ctrls += [overwrite_step, overwrite_switch, overwrite_width, overwrite_height, overwrite_vary_strength]
ctrls += [overwrite_upscale_strength, mixing_image_prompt_and_vary_upscale, mixing_image_prompt_and_inpaint]
ctrls += [debugging_cn_preprocessor, skipping_cn_preprocessor, canny_low_threshold, canny_high_threshold]
ctrls += [refiner_swap_method, controlnet_softness]
ctrls += freeu_ctrls
ctrls += inpaint_ctrls
state_is_generating = gr.State(False)
if not args_manager.args.disable_metadata:
ctrls += [save_metadata_to_images, metadata_scheme]
ctrls += ip_ctrls
def parse_meta(raw_prompt_txt, is_generating):
loaded_json = None
try:
if '{' in raw_prompt_txt:
if '}' in raw_prompt_txt:
if ':' in raw_prompt_txt:
loaded_json = json.loads(raw_prompt_txt)
assert isinstance(loaded_json, dict)
except:
loaded_json = None
if is_json(raw_prompt_txt):
loaded_json = json.loads(raw_prompt_txt)
if loaded_json is None:
if is_generating:
@ -553,41 +704,40 @@ with shared.gradio_root:
prompt.input(parse_meta, inputs=[prompt, state_is_generating], outputs=[prompt, generate_button, load_parameter_button], queue=False, show_progress=False)
load_parameter_button.click(modules.meta_parser.load_parameter_button_click, inputs=[prompt, state_is_generating], outputs=[
advanced_checkbox,
image_number,
prompt,
negative_prompt,
style_selections,
performance_selection,
aspect_ratios_selection,
overwrite_width,
overwrite_height,
sharpness,
guidance_scale,
adm_scaler_positive,
adm_scaler_negative,
adm_scaler_end,
base_model,
refiner_model,
refiner_switch,
sampler_name,
scheduler_name,
seed_random,
image_seed,
generate_button,
load_parameter_button
] + lora_ctrls, queue=False, show_progress=False)
load_parameter_button.click(modules.meta_parser.load_parameter_button_click, inputs=[prompt, state_is_generating], outputs=load_data_outputs, queue=False, show_progress=False)
def trigger_metadata_import(filepath, state_is_generating):
parameters, metadata_scheme = modules.meta_parser.read_info_from_image(filepath)
if parameters is None:
print('Could not find metadata in the image!')
parsed_parameters = {}
else:
metadata_parser = modules.meta_parser.get_metadata_parser(metadata_scheme)
parsed_parameters = metadata_parser.parse_json(parameters)
return modules.meta_parser.load_parameter_button_click(parsed_parameters, state_is_generating)
metadata_import_button.click(trigger_metadata_import, inputs=[metadata_input_image, state_is_generating], outputs=load_data_outputs, queue=False, show_progress=True) \
.then(style_sorter.sort_styles, inputs=style_selections, outputs=style_selections, queue=False, show_progress=False)
generate_button.click(lambda: (gr.update(visible=True, interactive=True), gr.update(visible=True, interactive=True), gr.update(visible=False, interactive=False), [], True),
outputs=[stop_button, skip_button, generate_button, gallery, state_is_generating]) \
.then(fn=refresh_seed, inputs=[seed_random, image_seed], outputs=image_seed) \
.then(advanced_parameters.set_all_advanced_parameters, inputs=adps) \
.then(fn=generate_clicked, inputs=ctrls, outputs=[progress_html, progress_window, progress_gallery, gallery]) \
.then(fn=get_task, inputs=ctrls, outputs=currentTask) \
.then(fn=generate_clicked, inputs=currentTask, outputs=[progress_html, progress_window, progress_gallery, gallery]) \
.then(lambda: (gr.update(visible=True, interactive=True), gr.update(visible=False, interactive=False), gr.update(visible=False, interactive=False), False),
outputs=[generate_button, stop_button, skip_button, state_is_generating]) \
.then(fn=update_history_link, outputs=history_link) \
.then(fn=lambda: None, _js='playNotification').then(fn=lambda: None, _js='refresh_grid_delayed')
reset_button.click(lambda: [worker.AsyncTask(args=[]), False, gr.update(visible=True, interactive=True)] +
[gr.update(visible=False)] * 6 +
[gr.update(visible=True, value=[])],
outputs=[currentTask, state_is_generating, generate_button,
reset_button, stop_button, skip_button,
progress_html, progress_window, progress_gallery, gallery],
queue=False)
for notification_file in ['notification.ogg', 'notification.mp3']:
if os.path.exists(notification_file):
gr.Audio(interactive=False, value=notification_file, elem_id='audio_notification', visible=False)
@ -605,6 +755,15 @@ with shared.gradio_root:
desc_btn.click(trigger_describe, inputs=[desc_method, desc_input_image],
outputs=[prompt, style_selections], show_progress=True, queue=True)
if args_manager.args.enable_describe_uov_image:
def trigger_uov_describe(mode, img, prompt):
# keep prompt if not empty
if prompt == '':
return trigger_describe(mode, img)
return gr.update(), gr.update()
uov_input_image.upload(trigger_uov_describe, inputs=[desc_method, uov_input_image, prompt],
outputs=[prompt, style_selections], show_progress=True, queue=True)
def dump_default_english_config():
from modules.localization import dump_english_config
@ -618,6 +777,7 @@ shared.gradio_root.launch(
server_name=args_manager.args.listen,
server_port=args_manager.args.port,
share=args_manager.args.share,
auth=check_auth if args_manager.args.share and auth_enabled else None,
auth=check_auth if (args_manager.args.share or args_manager.args.listen) and auth_enabled else None,
allowed_paths=[modules.config.path_outputs],
blocked_paths=[constants.AUTH_FILENAME]
)

8
wildcards/.gitignore vendored Normal file
View File

@ -0,0 +1,8 @@
*.txt
!animal.txt
!artist.txt
!color.txt
!color_flower.txt
!extended-color.txt
!flower.txt
!nationality.txt

100
wildcards/animal.txt Normal file
View File

@ -0,0 +1,100 @@
Alligator
Ant
Antelope
Armadillo
Badger
Bat
Bear
Beaver
Bison
Boar
Bobcat
Bull
Camel
Chameleon
Cheetah
Chicken
Chihuahua
Chimpanzee
Chinchilla
Chipmunk
Komodo Dragon
Cow
Coyote
Crocodile
Crow
Deer
Dinosaur
Dolphin
Donkey
Duck
Eagle
Eel
Elephant
Elk
Emu
Falcon
Ferret
Flamingo
Flying Squirrel
Giraffe
Goose
Guinea pig
Hawk
Hedgehog
Hippopotamus
Horse
Hummingbird
Hyena
Jackal
Jaguar
Jellyfish
Kangaroo
King Cobra
Koala bear
Leopard
Lion
Lizard
Magpie
Marten
Meerkat
Mole
Monkey
Moose
Mouse
Octopus
Okapi
Orangutan
Ostrich
Otter
Owl
Panda
Pangolin
Panther
Penguin
Pig
Porcupine
Possum
Puma
Quokka
Rabbit
Raccoon
Raven
Reindeer
Rhinoceros
Seal
Shark
Sheep
Snail
Snake
Sparrow
Spider
Squirrel
Swallow
Tiger
Walrus
Whale
Wolf
Wombat
Yak
Zebra