Have you guys had any success creating 2D style images from your Daz Studio renders?

DiasporaDiaspora Posts: 469

Hello everyone, I really want to dabble in 2d style animation, specifically pixel art (think mid 90s Sierra adventure games) and 90s - 00s disney style 2d animation (think Aladdin - Tarzan) by making 3d footage in Daz and then having an algorithm create 2d footage in the aforementioned target styles from the 3d footage I feed into it.

I'm personally not expecting miracles, so to speak, if the algorithm/AI/whatever you want to call it could get me "70-80% of the way there" and then I can retouch the frames by hand the rest of the way, that's more than good enough. Also, what would be really good would be if the tool's output would break up the elements into layers in ideally a PSD, though I can possibly make do without that.

I imagine I'm not the only one to be seeing the AI boom and thinking about how it could expedite some of the grunt work of creating illustrative style footage and imagery from our 3d renders, has anyone else gone down this particular rabbit hole?

Comments

  • SnowSultanSnowSultan Posts: 3,681

    I've experimented a lot with making 2D images from 3D renders, but not with animation. I suppose it's possible, but I do not know anything about generating AI video. 

    I have gotten good results, but as you said, those results will definitely require postwork and manual redrawing of some portions. As far as I know, it will not separate the elements into layers, but you can (and will need to) generate some parts separately, which can then be layered and masked (for example, you might want to use the eyes from one generation and the mouth from another).

    This image of mine was a 3D render, you can see it in the second little box below the main image.  https://www.deviantart.com/snowsultan/art/Too-Late-AI-postwork-over-3D-1112094522

    Anyway, you'll need to have a local version of Stable Diffusion with Controlnets, Illustrious or Pony models/loras, and a lot of patience. Hope that's of some help at least.   :)

  • DiasporaDiaspora Posts: 469
    edited April 17

    "I've experimented a lot with making 2D images from 3D renders, but not with animation. I suppose it's possible, but I do not know anything about generating AI video. "

    I mean, at least if I just want to dabble in making a short animation, a few seconds at 8FPS, then just converting frame by frame is enough. 

    But if this works out and I want to start making serious amounts of content in this workflow then hopefully a more streamline process will be attainable.

    Anyway, with that said, thank you for the lead!

    I took a look at your image and you seem to be on the right track (though I personally am not chasing the anime face style at this time)

    Post edited by Diaspora on
  • SnowSultanSnowSultan Posts: 3,681
    edited April 17

    People are making short AI videos using a Chinese model (name escapes me at the moment) and turning real footage into anime or toon-styled clips, so I'd imagine it's actually very likely possible to make what you want if you can animate something long enough in Studio.

    Sure, I wasn't really aiming for an anime look there either, I just hadn't figured out how to use other styles all that well at the time. I do kinda like her goofy face there though.   ;)  I haven't really posted many other of my tests, although you might want to look at my image 'Blonde Bop Reborn". That was also a VERY old 3D render run through a semi-Disney style AI model and then I drew over a lot of it.

    Post edited by SnowSultan on
  • DiasporaDiaspora Posts: 469
    edited April 17

    SnowSultan, looking at your images, Blonde Bop Reborn definitely looks very good and I respect the touch ups you did especially with the hand drawn effect on the hair but subjectively I feel like it has the issue I often have with AI assisted or whole generated images which is that "I feel I've already seen this before" kind of look, to put it very simply. That's not a knock on you, you crafted it well but do you know what I mean?

    I say all this because I compare it to your other image 2D Cutie, which essentially you 'grease penciled' over an AI generated image and to your credit your final product really nailed that sort of, what would I call it, she looks like a character from the movie The Iron Giant. I wish I could make animations that look persuasively in that style. The image just seems to have a sort of flow and natural gestalt that is a real struggle to attain for someone like me who isn't the best illustrator. 

    But if Stable Diffusion fed by my 3d footage can get me 80% of the way there, then I think I would personally be up to getting good enough at illustrating to get the remainder 20% of the way by hand. 

    Anyway, thank you for your insights and for sharing, cool gallery overall by the way. 

    Post edited by Diaspora on
  • CybersoxCybersox Posts: 9,126

    SnowSultan said:

    People are making short AI videos using a Chinese model (name escapes me at the moment) and turning real footage into anime or toon-styled clips, so I'd imagine it's actually very likely possible to make what you want if you can animate something long enough in Studio.

     

    If you're thinking about the same person that I am, her name is Barbin and she's not a model but a mime who frequently wears lolita/anime style clothing while doing a mechanical girl act.  Most of those videos are made using TikTok aps that basically rotoscope the motion and then apply a cel-look filter.  

    Really, it seems to me that DAZ Studio is the wrong application to try to use as a basis for something like this, though DAZ 3D assets could certainly be exported to a different program that's better set up to do classic 2D animation like Blender with it's Greasepencil, onion-skininng functions, and ever popular free price tag.  

  • wolf359wolf359 Posts: 3,867
    edited April 18

    if you google 'Ghibli Style animation with AI"

    you will see this is very possible without needing a 3D program
     

    Post edited by wolf359 on
  • CybersoxCybersox Posts: 9,126
    edited April 18

     

    Post edited by Cybersox on
  • CybersoxCybersox Posts: 9,126
    edited April 18

    duplicate post

    Post edited by Cybersox on
  • CybersoxCybersox Posts: 9,126

    SnowSultan said:

    I've experimented a lot with making 2D images from 3D renders, but not with animation. I suppose it's possible, but I do not know anything about generating AI video. 

    I have gotten good results, but as you said, those results will definitely require postwork and manual redrawing of some portions. As far as I know, it will not separate the elements into layers, but you can (and will need to) generate some parts separately, which can then be layered and masked (for example, you might want to use the eyes from one generation and the mouth from another).

     

    AI has become such a slippery slope, as there so much released these days that has at least a little AI under the hood.  That said, there's no doubt that there are some tedious tasks that it can handle that would be insanely time consuming to do any other way.  I think that DeviantArt as more or less the right idea when they allow the use of images that use AI without the required labeling of them as AI created, as long as the majority of the image was "self-created".  New Label Requirement for AI Artwork by team on DeviantArt 

  • SnowSultanSnowSultan Posts: 3,681

    If you're thinking about the same person that I am, her name is Barbin and she's not a model but a mime who frequently wears lolita/anime style clothing while doing a mechanical girl act.

    Sorry, when I said "model", I meant an AI model, not a person.

     

    but subjectively I feel like it has the issue I often have with AI assisted or whole generated images which is that "I feel I've already seen this before" kind of look, to put it very simply. 

    Sure, I know that a lot of AI art looks the same. The reason why 2D Cutie probably looks different is that I redrew that completely, where I just edited Blonde Bop Reborn's AI result (very heavily though).

    Blender does have better tools for rotoscoping and drawing traditonally over 3D renders than most programs, but it won't save you nearly as much work as fixing up AI results will.

  • CybersoxCybersox Posts: 9,126
    edited April 19

    SnowSultan said:

    Blender does have better tools for rotoscoping and drawing traditonally over 3D renders than most programs, but it won't save you nearly as much work as fixing up AI results will.

    Depends on which end of the workflow you're looking at.  You can buy or rent a pretty good motion capture setup for very little these days and do just about anything you want with that.  Going with all AI, on the other hand, comes with the nebulous legality of the process hanging over the whole thing and the definite possibility that anything that you do with it may end up not being distributable or copyrightable at some point.  That may seem a bit paranoid, but it's a fear that gained real teeth with ChatAI's recent blatant duplicaton of the Studio Ghibli artstyle in recent weeks, as even though Studio Ghibli hasn't, as of this writing, taken any legal actions (current C&D letters floating around the net all being fakes to the best of my knowledge), there are few who've tried to argue that it isn't actionable given that it obviously required the ingestion of a large volume of copyrighted work to even exist, and that ChatAI (now a for-profit venture) was blatant enough to actually use another company's trademarked brand name... surprise  

    Personally, I'm restricting my use of AI to final cleanup work and the generation of singular elements until we see a real legal case that sets some kind of rules as to how much AI is too much AI to be considered my own work anymore.      

     

    Post edited by Cybersox on
  • DiasporaDiaspora Posts: 469

    Just to reiterate, I AM talking about just using an AI tool as just a link in the chain, so to speak. 

  • 3Diva3Diva Posts: 11,956

    I highly recommend animating with Filament or Filatoon if you're going to animate in Daz Studio as the rendering is much faster than Iray or 3Delight. 

    I don't know what kind of 2D style you're after, but my Easy FilaToon product can help people create a variety of NPR styles: https://www.daz3d.com/easy-filatoon

    You could experiment with the default Filatoon shaders that's included for free in Daz Studio to see if you enjoy rendering with Filament and Filatoon first. 

  • DiasporaDiaspora Posts: 469

    I think filatoon might be the better approach for the time being. 3Diva, I'll buy your product soon because it sounds spot on for what I need.

  • wsterdanwsterdan Posts: 2,612

    Diaspora said:

    I think filatoon might be the better approach for the time being. 3Diva, I'll buy your product soon because it sounds spot on for what I need.

    I picked it up when it came out and now that I can render Filatoon on the Mac, I actually used it today! It worked perfectly. I've been trying to get a nice. consistent  2D look out of DAZ for decades, and while pwToon came close, it was too slow and tended to give me varying qualities (I found I had to have different settings if a character was closer to or further away from the camera, etc.). Easy Filatoon gave me better results toon-wise in the first hour I used it than I got out of pwToon after months of usage. 

    I'm still experimenting, but this is what I got pretty much on my first try; here's my first two attempts at a 2D-style animation using her product on the character, clothing and starship bridge.

    NOTE: the forward-facing animation used a lip sync pose I made years ago with DAZ's 32-bit lip sync, the second one is my very first attempt at hand-keying the lip sync, head movements and eye blinks. Man, soooo much practice is needed!

    https://sterdan.com/wp-content/uploads/2025/04/Captain-Corbett-Opening.mp4

    https://sterdan.com/wp-content/uploads/2025/04/Capt-Knight-Not-to-me.mp4

  • WendyLuvsCatzWendyLuvsCatz Posts: 38,870

    wsterdan said:

    Diaspora said:

    I think filatoon might be the better approach for the time being. 3Diva, I'll buy your product soon because it sounds spot on for what I need.

    I picked it up when it came out and now that I can render Filatoon on the Mac, I actually used it today! It worked perfectly. I've been trying to get a nice. consistent  2D look out of DAZ for decades, and while pwToon came close, it was too slow and tended to give me varying qualities (I found I had to have different settings if a character was closer to or further away from the camera, etc.). Easy Filatoon gave me better results toon-wise in the first hour I used it than I got out of pwToon after months of usage. 

    I'm still experimenting, but this is what I got pretty much on my first try; here's my first two attempts at a 2D-style animation using her product on the character, clothing and starship bridge.

    NOTE: the forward-facing animation used a lip sync pose I made years ago with DAZ's 32-bit lip sync, the second one is my very first attempt at hand-keying the lip sync, head movements and eye blinks. Man, soooo much practice is needed!

    https://sterdan.com/wp-content/uploads/2025/04/Captain-Corbett-Opening.mp4

    https://sterdan.com/wp-content/uploads/2025/04/Capt-Knight-Not-to-me.mp4

    very tidy 

  • SapphireBlueSapphireBlue Posts: 1,207

    wsterdan That's awesome! I was going to ask you whether you could post your progress in the forums - happy to see you got things working on your Mac now to test things out with the new alpha release. Filatoon and animation in FilaToon are indeed so quick and easy to get started with, and 3Diva's Easy FilaToon makes it even easier. Hope you post more as you get things going, in the FilaToon threads or the animation threads etc. It's cool to see everyone's experiments with animating in FilaToon. smiley

Sign In or Register to comment.