Code

Added flux demo#3418

Code

Open

cehongwang wants to merge 30 commits intomainpytorch/TensorRT:mainfrom

flux-demopytorch/TensorRT:flux-demoCopy head branch name to clipboard

Commits

Commits on Apr 7, 2025

• Added CPU offloading

  cehongwang

  committed
  f85820c

  Copy full SHA for f85820c

• Chagned CPU offload to default

  cehongwang

  committed
  4242743

  Copy full SHA for 4242743

• Added support to module with graph break

  cehongwang

  committed
  e87d27e

  Copy full SHA for e87d27e

• Added back the control flag and fixed the CI

  cehongwang

  committed
  76cab94

  Copy full SHA for 76cab94

• Chagned CPU offload to default

  cehongwang

  committed
  6352110

  Copy full SHA for 6352110

• Added flux demo

  cehongwang

  committed
  214e2e6

  Copy full SHA for 214e2e6

• changed the file place and deleted unnecessary code

  cehongwang

  committed
  c9d8456

  Copy full SHA for c9d8456

• Fixed memory overhead and enabled Flux with Mutable Module

  cehongwang

  committed
  e77737d

  Copy full SHA for e77737d

• Supported LoRA

  cehongwang

  committed
  42c384d

  Copy full SHA for 42c384d

• Refined Flux demo, solved a bug of device mismatch, and prototyped CudaGraph and Weight streaming

  cehongwang

  committed
  33db1cb

  Copy full SHA for 33db1cb

• Enabled Cuda Graph

  cehongwang

  committed
  c69f41c

  Copy full SHA for c69f41c

• Enabled weight streaming and CudaGraph. Supported MTTM saving with dynamic shapes.

  cehongwang

  committed
  8f44d7f

  Copy full SHA for 8f44d7f

• Changed the Refitting test to disable CPU offload

  cehongwang

  committed
  044f4e6

  Copy full SHA for 044f4e6

• Fixed Cuda Error

  cehongwang

  committed
  d383be4

  Copy full SHA for d383be4

• Fixed the bug of SDXL Cuda Error

  cehongwang

  committed
  580fc03

  Copy full SHA for 580fc03

• Changed the way to enable CudaGraph for MTTM

  cehongwang

  committed
  3e8323f

  Copy full SHA for 3e8323f

• Finalize the refit revision

  cehongwang

  committed
  92ae47d

  Copy full SHA for 92ae47d

• Fixed the comments

  cehongwang

  committed
  98cbd76

  Copy full SHA for 98cbd76

• Correct the flux export example

  cehongwang

  committed
  39ac60e

  Copy full SHA for 39ac60e

• Added a textbox to display time the generation process takes

  cehongwang

  committed
  6caf833

  Copy full SHA for 6caf833

• Added perf script

  cehongwang

  committed
  c018151

  Copy full SHA for c018151

Commits on Apr 9, 2025

• added back control flag

  cehongwang

  committed
  9e390da

  Copy full SHA for 9e390da

Commits on Apr 18, 2025

• trying to add quantization to Flux

  cehongwang

  committed
  ba76f6d

  Copy full SHA for ba76f6d

Commits on Apr 24, 2025

• Enable int8 and fp8 quantization for FLUX

  cehongwang

  committed
  27dee53

  Copy full SHA for 27dee53

Commits on Apr 25, 2025

• Optimized FLUX compilation memory usage

  cehongwang

  committed
  044acdf

  Copy full SHA for 044acdf

Commits on May 5, 2025

• Optimized lowering and decomposition to benchmark quantization again

  cehongwang

  committed
  b06e632

  Copy full SHA for b06e632

Commits on May 9, 2025

• Fixed the benchmark typo

  cehongwang

  committed
  3dcf128

  Copy full SHA for 3dcf128

• Use MutableTorchTensorRTModule to do quantization

  cehongwang

  committed
  1581f0c

  Copy full SHA for 1581f0c

Commits on May 12, 2025

• Added quantization debug script

  cehongwang

  committed
  18b6455

  Copy full SHA for 18b6455

Commits on May 20, 2025

• Fixed fp16 quantization error

  cehongwang

  committed
  a4ff6bb

  Copy full SHA for a4ff6bb