ComfyUI is an open source, node-based, generative artificial intelligence computer program that allows users to generate images from a series of text prompts. It uses Stable Diffusion as the base model for its image capabilities combined with other tools such as ControlNet and LCM Low-rank adaptation with each tool being a node in the program.[3]
| Original author(s) | comfyanonymous |
|---|---|
| Initial release | January 16, 2023[1] |
| Repository | github |
| Written in | Python |
| License | GPLv3[2] |
| Website | www |
History
editComfyUI was released on GitHub in January 2023. According to comfyanonymous, the creator, a major goal of the project was to improve on existing software designs in terms of the user interface.[4] The creator had been involved with Stability AI but by 3 June 2024 that involvement had ended and an organization called Comfy Org had been created along with the core developers.[5] In July 2024, Nvidia announced support for ComfyUI within its RTX Remix modding software.[6] As of July 2024, the project has 41.8k stars on GitHub.[7]
Features
editComfyUI's main feature is that it is node based. Each node has a function such as "load a model" or "write a prompt".[8] A default node group is also included with the program.[8] When a prompt is queued, a highlighted frame appears around the currently executing node, starting from "load checkpoint" and ending with the final image and it's save location.[8] Workflows can be saved to a file,[9] allowing users to re-use node workflows and share them with other users.[10] The file format for the workflows is in JSON and can be embedded in the generated images.[11] Users have also created custom extensions to the base system[3] such as for AnimateDiff, which aims to create videos.[12]
LLMVision Extension Compromise
editIn June 2024, a hacker group called "Nullbulge" compromised an extension of ComfyUI to add malicious code to it.[13] The compromised extension, called ComfyUI_LLMVISION, was used for integrating the interface with AI language models GPT-4 and Claude 3, and was hosted on GitHub. Nullbulge hosted a list of hundreds of ComfyUI users' login details across multiple services on its website, while users of the extension reported receiving numerous login notifications. vpnMentor conducted security research on the extension and claimed it could "steal crypto wallets, screenshot the user’s screen, expose device information and IP addresses, and steal files that contain certain keywords or extensions".
Nullbulge's website claims they targeted users who committed "one of our sins", which included AI-art generation, art theft, promoting cryptocurrency, and any other kind of theft from artists such as from Patreon. They claimed that they were "a collective of individuals who believe in the importance of protecting artists' rights and ensuring fair compensation for their work" and that they believed that "AI-generated artwork is detrimental to the creative industry and should be discouraged".[13]
References
edit- ^ comfyanonymous. "Initial commit". github. Retrieved 10 July 2024.
- ^ comfyanonymous. "LICENSE". github. Retrieved 10 July 2024.
- ^ a b 机器之心 (16 November 2023). "当韩国女团BLACKPINK进军二次元，清华叉院AI神器原来还能这么玩-36氪". 36氪 (in Chinese).
- ^ comfyanonymous (18 May 2023). "ComfyUI is now 4 months old!". ComfyUI blog. Retrieved 11 July 2024.
- ^ "ComfyUI 作者团队成立 Comfy Org- DoNews快讯". DoNews.
- ^ Harper, Christopher (4 July 2024). "Nvidia's RTX Remix goes open source —chipmaker adds Rest API to interface with ComfyUI for AI remastering or generating new graphics in real time". Tom's Hardware. Retrieved 11 July 2024.
- ^ comfyanonymous. "ComfyUI". github. Retrieved 10 July 2024.
- ^ a b c 田口, 和裕. "画像生成AI「Stable Diffusion」使い倒すならコレ！ 「ComfyUI」基本の使い方 (1/3)". ascii.jp (in Japanese).
- ^ 田口, 和裕 (27 March 2024). "Macで始める画像生成AI 「Stable Diffusion」ComfyUIの使い方 (3/5)". ascii.jp (in Japanese).
- ^ 白鲸出海 (23 May 2024). "一家成都游戏公司，做出了一款千万月访问量的AI图像产品-36氪". 36氪 (in Chinese).
- ^ しらいはかせ (18 December 2023). "画像生成AIを使い倒す！「Stability Matrix」で使えるWebUIを紹介【生成AIストリーム】". Impress Watch (in Japanese).
- ^ 新, 清士. "アニメの常識、画像生成AIが変える可能性「AnimateDiff」のすごい進化". ascii.jp (in Japanese).
- ^ a b Maiberg, Emanuel (2024-06-11). "Hackers Target AI Users With Malicious Stable Diffusion Tool on GitHub to Protest 'Art Theft'". 404 Media. Retrieved 2024-06-14.