vs设置网站开发环境,网络营销运营推广,张家港那家做网站,电子商务和市场营销的区别一、写在前面
按常理#xff0c;这里应该长篇大论地介绍一下Sora发布对各行业各方面产生的影响。不过#xff0c;这类文章已经很多了#xff0c;我们今天主要聊聊那些已经成熟的解决方案、那些已经可以“信手拈来”的成果#xff0c;并以此为基础#xff0c;看看Sora发布…一、写在前面
按常理这里应该长篇大论地介绍一下Sora发布对各行业各方面产生的影响。不过这类文章已经很多了我们今天主要聊聊那些已经成熟的解决方案、那些已经可以“信手拈来”的成果并以此为基础看看Sora发布后的时代我们能做些什么。
本文涉及的、可尝试的应用地址
[1] Stable Video Diffusion huggingface space仅支持图生视频https://huggingface.co/spaces/multimodalart/stable-video-diffusion [2] 小诺AI支持文生视频和图生视频微信搜索小程序-小诺AI [3] Pika支持文生视频https://pika.art/ [4] Open-Sora支持文生视频GitHub - hpcaitech/Open-Sora: Open-Sora: Democratizing Efficient Video Production for All
二、当前视频生成解决方案一览
最近看了很多文生视频的开源构建项目[1]、[4]以及闭源直接应用项目[2]、[3]一个直观感受是大家对Sora的应用都很期待。Sora发布至今仅一个月已经有很多可以直接体验的文生视频产品被推出这里选择几个比较有人气的应用介绍下。
pika
Pika[3]是Sora前产物它们勇敢地做出了自己的尝试。虽然Sora发布后对Pika的质疑声音越来越大但毋庸置疑Pika是先行者而且取得了不错的成绩而且Pika也在迭代优化中有了Sora的经验Pika有可能在之后带给人们更大的惊喜个人将保持期待。
这里给出一个官网的示例。 Prompt3d animation, a cute boy is standing in a house, spring festival interior, lunar new year, holiday.
Stable Video Diffusion
Stable Video Diffusion[1]也是Sora前产物与Pika不同的是SVD是一个开源项目。所以有一种观点是Sora借鉴了SVD这种观点有一定的道理。查看Sora的report有很多细节如DiT与SVD是一致的。而且SVD在前说借鉴也没问题。个人觉得SVD是目前视频生成质量最高的解决方案它能接受的输入形式是image。这里是一个示例输入的image是一个静态的发射中的火箭而输出的视频将整个发射过程动态化呈现。 图片输入 rocket 视频输出 rocket博客不能放视频这里是将video抽帧呈现的gif
我们探讨一个问题SVD的执行过程因为缺少语义的指导所以它的任务只是将其“动态化”这种“动态化”有可能是违背物理规律的。通俗地讲上面的火箭也有可能是水平运动的我自己的尝试中出现过这种情况并没有一种途径去指定image中的物体的具体轨迹指定如何运动才是正确的这里的运动形式是单纯地由模型在数据集中学习的。
SVD的后续工作应该是加入text提供更清晰的语义。
Open-Sora
Open-Sora[4]是Sora后产物是完全根据Sora公布的report复现的开源项目。以下引自Open-Sora项目首页的简介。
Open-Sora项目是一项致力于高效制作高质量视频并使所有人都能使用其模型、工具和内容的计划。 通过采用开源原则Open-Sora 不仅实现了先进视频生成技术的低成本普及还提供了一个精简且用户友好的方案简化了视频制作的复杂性。 通过 Open-Sora我们希望更多开发者一起探索内容创作领域的创新、创造和包容。Open-Sora 项目目前处在早期阶段并将持续更新。
这里也给出一些示例直观感受下效果。 PromptA serene night scene in a forested area. The first frame shows a tranquil lake reflecting the star-filled sky above. The second frame reveals a beautiful sunset, casting a warm glow over the landscape. The third frame showcases the night sky, filled with stars and a vibrant Milky Way galaxy. The video is a time-lapse, capturing the transition from day to night, with the lake and forest serving as a constant backdrop. The style of the video is naturalistic, emphasizing the beauty of the night sky and the peacefulness of the forest. PromptA soaring drone footage captures the majestic beauty of a coastal cliff, its red and yellow stratified rock faces rich in color and against the vibrant turquoise of the sea. Seabirds can be seen taking flight around the cliffs precipices. As the drone slowly moves from different angles, the changing sunlight casts shifting shadows that highlight the rugged textures of the cliff and the surrounding calm sea. The water gently laps at the rock base and the greenery that clings to the top of the cliff, and the scene gives a sense of peaceful isolation at the fringes of the ocean. The video captures the essence of pristine natural beauty untouched by human structures. PromptThe video captures the majestic beauty of a waterfall cascading down a cliff into a serene lake. The waterfall, with its powerful flow, is the central focus of the video. The surrounding landscape is lush and green, with trees and foliage adding to the natural beauty of the scene. The camera angle provides a birds eye view of the waterfall, allowing viewers to appreciate the full height and grandeur of the waterfall. The video is a stunning representation of natures power and beauty. PromptA serene underwater scene featuring a sea turtle swimming through a coral reef. The turtle, with its greenish-brown shell, is the main focus of the video, swimming gracefully towards the right side of the frame. The coral reef, teeming with life, is visible in the background, providing a vibrant and colorful backdrop to the turtles journey. Several small fish, darting around the turtle, add a sense of movement and dynamism to the scene. The video is shot from a slightly elevated angle, providing a comprehensive view of the turtles surroundings. The overall style of the video is calm and peaceful, capturing the beauty and tranquility of the underwater world.
小诺AI
小诺AI[3]是舒笔科技推出的一款微信小程序是基于SVD优化的文生视频产品。在SVD的基础上小诺AI集成了支持文字Prompt输入的功能打通了文生视频的pipeline使用户所写即所得。
该产品目前只支持英文输入的功能不过没关系小诺AI同时集成了Prompt咒语生成功能可将中文Prompt翻译。同时如果也可以利用该功能对Prompt进行扩写。
这里是一些示例的展示。 PromptThe sun is setting by the mountain. Promptbreathtaking selfie photograph of astronaut floating in space, earth in the background. award-winning, professional, highly detailed Promptbreathtaking night street of city, neon lights. award-winning, professional, highly detailed Promptanime artwork an empty classroom. anime style, key visual, vibrant, studio anime, highly detailed Prompta beautiful room Promptanime artwork an island surrounding by the sea, dramatic, anime style, key visual, vibrant, studio anime, highly detailed Promptconcept art of a warrior with a sword, clouds. digital artwork, illustrative, painterly, matte painting, highly detailed, cinematic composition Prompt16-bit pixel art, a cozy cafe side view, a beautiful day
