/robowaifu/ - DIY Robot Wives

Advancing robotics to a point where anime catgrill meidos in tiny miniskirts are a reality.

LynxChan updated to 2.5.7, let me know whether there are any issues (admin at j dot w).


Reports of my death have been greatly overestimiste.

Still trying to get done with some IRL work, but should be able to update some stuff soon.

#WEALWAYSWIN

Max message length: 6144

Drag files to upload or
click here to select them

Maximum 5 files / Maximum size: 20.00 MB

More

(used to delete files and postings)


Welcome to /robowaifu/, the exotic AI tavern where intrepid adventurers gather to swap loot & old war stories...


Open file (485.35 KB 1053x1400 0705060114258_01_Joybot.jpg)
Robowaifu Simulator Robowaifu Technician 09/12/2019 (Thu) 03:07:13 No.155
What would be a good RW simulator. I guess I'd like to start with some type of PCG solution that just builds environments to start with and build from there up to characters.

It would be nice if the system wasn't just pre-canned, hard-coded assets and behaviors but was instead a true simulator system. EG, write robotics control software code that can actually calculate mechanics, kinematics, collisions, etc., and have that work correctly inside the basic simulation framework first with an eye to eventually integrating it into IRL Robowaifu mechatronic systems with little modifications. Sort of like the OpenAI Gym concept but for waifubots.
https ://gym.openai.com/
While attempting to run a physics simulation on a character model I noticed the physics behaved really strangely for body parts. It was treating them more like balls since it doesn't take into account the volume of objects. It was a bit too oversimplified to simulate adequate rag doll physics, so I played around with creating rigid bodies of varying mass by joining multiple bodies together with hinge joints using an angular upper and lower bound of 0. There were some issues at first but once I turned the physics fps up to 600 the torque and everything worked as expected, so I tried applying the idea to meshes. I can't go into a tutorial right now but I'll leave some notes here of what I did. While separating a mesh in edit mode of Blender, go to Mesh > Convex Hull to preview the collision boundary of the area you're separating. After separating a piece make sure to reset the origin of the mesh to its center with Ctrl+Alt+X > Origin to Geometry, or else you'll have to reimport and rerig the entire model to fix any mistakes. In Blender you can enable the 3DPrint add-on to calculate the volume of meshes. Once a collision mesh is separated into several meshes you can divide up the total mass among their volumes and have much more realistic physics. If you forget a hinge joint for intersecting pieces of a mesh, they will behave strangely trying to separate themselves. The physics in general seem to behave strangely above 600 fps. 120-240 fps seemed to work best for my experiments. You can download the statue and Godot project here: https://files.catbox.moe/px22po.xz And the 25 mb reindeer statue material if you really want it: https://files.catbox.moe/te0qfa.xz Original statue model from here: https://sketchfab.com/3d-models/reindeer-head-statue-c8099ccd38364f68b26bf552cc461f00 I'm not really satisfied with the physics but it's better than nothing. The simulation just has to be useful enough to give AI an advantage before training in real life and a place for us to prototype ideas. I'll see how well it does once I finish the claw crane. Once that's working I'll train AI with hindsight experience replay to learn how to control it. I also found some helpful command line options for running simulations as fast as possible while using Pytorch in Godot without the headless server edition: --disable-render-loop Disable render loop so rendering only occurs when called explicitly from script. --fixed-fps <fps> Force a fixed number of frames per second. This setting disables real-time synchronization. --time-scale <scale> Force time scale (higher values are faster, 1.0 is normal speed). Although I'm thinking it'll be far easier and better to use Pybullet with all the extra features for robots without all the overhead of Godot. I feel Godot will be better for demos and releases than developing AI and running complex physics simulations.
>>5446 Thanks, that's very interesting. I thought more to use simple simulations to see how strong my motors and muscles need to be, not so much for training her to use her body while simulating it completely. James Bruton showed how he trained his Robot X by trial an error to walk. I linked the video here: >>5358
>>5447 Wow, this guy is prolific. I think the robot drifting off to the left could have been caused by different amounts of friction. I looked at his code and he's using a purely symbolic approach rather than neural networks. With neural networks noise can be introduced into the simulation to help robustify training. Under the motor section of hinge joints in Godot you can define the target velocity and max torque. Example: https://files.catbox.moe/dl3qd8.tscn It seems Godot armatures are extremely buggy and difficult to rig so I'm gonna switch to Pybullet instead. I found someone on Youtube using it to run simulations for a robot: https://www.youtube.com/channel/UCMw7ksxe5uW4DnBOdvxWWvQ >>5446 Just noticed I forgot to unlink the material for the reindeer statue before saving. If anyone has trouble running it without the material, just open reindeer statue.tscn and solid reindeer statue.tscn, load anyway, save and it will run.
>>5449 Also just noticed the physics fps affects how much torque is applied to joints in Godot. For the animation I had it set to 120 fps under Project Settings > Physics > Common > Physics Fps. At the default 60 fps they both fail to lift the cube.
>>5449 >>5450 Thanks, hope someone picks that up. I probably won't have time anytime soon. We'll see, I'm already curious about it. But I can't do everything at once...
I found some other software for simulating robots called Gazebo: http://gazebosim.org/ https://www.youtube.com/watch?v=JMVGSxT79rk It has over 200 package dependencies so use the auto installer: http://gazebosim.org/tutorials?cat=install There's a Gazebo model respository available here: https://github.com/osrf/gazebo_models Dump that shit in $HOME/.gazebo/models. It looks like they're in SDF format so they should be able to work with Bullet too. There is a Blender 2.8 plugin that exports SDF and URDF robot files which can used with Bullet: https://github.com/david0429/blender_gazebo But it requires ROS: https://www.ros.org/ But for whatever reason Gazebo doesn't work on my system and ROS is in dependency hell and uninstallable, so I'm gonna write a plugin to export Blender 2.8 models in SDF since there doesn't seem to be one. I sure as hell don't want my robowaifu or workflow running off 500+ dependencies. SDF library: https://github.com/osrf/sdformat SDF specification: http://sdformat.org/spec >>5451 Yeah, getting this shit to work is a fucking time sink. I'll do my best to create an easy solution. We'd get hell of a lot more people on board if things were easier.
Open file (524.22 KB 852x480 itwerks.mp4)
>>5453 Updated Godot's Better Collada Exporter to Blender 2.83: https://gitlab.com/kokubunji/collada-exporter-2.83 Needed to because SDF files are a combination of Collada and STL files. An extra benefit is that models import into Godot correctly now without their meshes and armatures getting rekt. Should be able to test walking in Godot now with proper joint constraints. WIP 2B ragdoll if anyone is bored: https://files.catbox.moe/4ufnns.xz
I was just wondering, why no one here seems to use OpenAI gym but tries to create their own simulators. Okay, I might understand it in cases where it's about simulating movement of a waifu model, because OAI gym might have another scope. Is it just that?
Open file (804.92 KB 1358x1560 1594910210402-2.png)
>>5491 >OpenAI gym Well, I'd guess it's a basic and well-deserved mistrust of Silicon Valley tech giants. They certainly aren't well-aligned with the basic tenets of /robowaifu/ in general. Having full control over every aspect of our systems is obviously a very high priority for us here in the end. No one really wants their waifus suddenly disabled by the Ministry of Truth yanking the rug out b/c wrongthink. But yeah, I believe I made an effort a couple of years or so ago, but couldn't get it working. I just had another look at it on their repo and maybe I can give it another go sometime. I think it's fine to prototype with closed system like (ironically) OpenAI produces, as long as the intent going in is simply investigation and inspiration only. Thanks for the reminder, Anon.
>>5491 OpenAI gym doesn't provide much except an easy platform to do reinforcement learning. If you decide to use it your model will be super glued to RL. I use it for testing RL ideas and like RL a lot but something just seems off about it. Life doesn't live trying to maximize its rewards. Most RL algorithms act extremely dumb because they're too greedy and can't solve games like Pitfall or Montezuma's Revenge. It's like it's stuck in a Chinese finger trap where the reward function is how far apart the fingers are. RL will pull its fingers with all its might, not realizing it will come off if it pushes them together and holds it from stretching. It may find the solution eventually through brute force but that has no value because it was purely random chance. People have tried to change the reward function to avoid this problem but then it does the same thing just in a different or more convoluted situation they didn't think about. RL has gotten something right but it seems to be grossly missing other dimensions of intelligence.
>>5494 >It's like it's stuck in a Chinese finger trap where the reward function is how far apart the fingers are. RL will pull its fingers with all its might, not realizing it will come off if it pushes them together and holds it from stretching. It may find the solution eventually through brute force but that has no value because it was purely random chance. Fabulous way to word that issue.
Open file (27.05 KB 711x399 Cameron-eyeopening.jpg)
>>5492 >>5494 Good that I asked, this was very interesting. Thanks. Though, I thought OpenAI is a foundation and their software is open source, so that part shouldn't be a problem.
>>5501 >Though, I thought OpenAI is a foundation and their software is open source, so that part shouldn't be a problem. They are a good example of the double-standard corruption going on in Silicon Valley. They're happy to receive tax-exempt, non-profit donations (to the tune of US$1Bn a pop https://web.archive.org/web/20190723065252/https://news.microsoft.com/2019/07/22/openai-forms-exclusive-computing-partnership-with-microsoft-to-build-new-azure-ai-supercomputing-technologies/ ), all the while keeping the source closed to `protect us` from dem ebil_nahdzees. It's literally George Orwell's Ministry of Truth alive and well today, but in a tail-wagging-the-dog form of it (ie, Industry tells the Government what to do). But don't get me started... XD
>>5501 OpenAI gym is open-source but the documentation is atrocious for custom environments. When I used gym retro to build a library for playing SNES games with AI easily there was a bug in some games with hi-res mode and absolutely no documentation to work with. After digging in the source code for awhile I found they buried the functionality of the emulator and there was only one dude working on it. For the billions of dollars they're receiving in funding you'd think they'd have more than one guy on open-source code. Similarly with the code they release for their papers they only release the bare minimum with no documentation rather than the actual full code they used in their paper. Another issue is I want to have full access and control over the simulation while prototyping parts. One of the reasons why I use PyTorch instead of Tensorflow is because it uses a dynamic graph for models, whereas Tensorflow's is static and cannot be changed while running. In PyTorch you can plug in and disconnect other networks at runtime and shutdown various parts of a network to only train specific parts. It's a lot more robust and gives more freedom with what you can do with it.
>>5504 >OpenAI gym is open-source but the documentation is atrocious for custom environments. Fair enough, my little rant >>5502 was about them sequestering the 'open' source code of GPT-3, and instead doling it out to only the chosen few and for a very hefty fee, no doubt. And even if they made access to a run-time solution for their GPT system entirely no-cost, I still don't consider that 'open' by our standards. You still wind up entirely beholden to their cloud-based system for your waifu to function. One wrong peep out of you on social media one day and they pull the plug. The only way around this I can think of is we somehow create our own models. And surely we can't be the only group desiring freedom from this type of commercial bondage by Big Tech. There must be literally hundreds (or thousands?) of groups interested in truly open conversational AI, and in a diverse set of fields as well. As things stand, IMO there is very little 'open' about what OpenAI is doing.
>>5506 The closest thing I know of is Lex Fridman's discord server. It has a lot of people interested in AI sharing and discussing various papers and their own projects. They're not really focused or collaborative though. I only know of a few people really innovating with AI on their own. Most of the discussions I see online are by beginners still learning the basics. It's an extremely difficult field to get into that requires time and money and it's constantly evolving. This is the reality we face against Big Tech. I think our best bet is to keep working on our projects and hope they attract the right people. My vision at the moment is to finish the Blender SDF add-on so people can quickly design robots in Blender, export them to Bullet, then drag and drop SDFs into the simulation window to automatically start training. This easy workflow should attract a lot of creative minds, and under the hood there could be different learning algorithms and parameters for people to tune. None of this 2 GB bloatware with 500+ dependencies. People who want to make robots can focus on robots and those who wanna make AI can make AI. Our efforts will synergize that way.
>>5510 Sounds like a good plan Anon. You plainly are ahead of me in keeping up with current hobbyist trends. I'd suggest you continue on with your plans. I consider you a leader in this, at least insofar as we on /robowaifu/ are concerned. Godspeed.
Open file (332.66 KB 1366x768 Workspace1_046.png)
>>5510 >>5511 BTW, I've got Godot successfully building today on my box, and hopefully I can soon be up to speed and follow along with you Anon. (3.2 branch b/c my little machine doesn't into Vulkan) https://github.com/godotengine/godot/tree/3.2 BTW there are good tutorials about the engine, etc., on the official Godot YT channel itself.
>>5510 If you ask about organized groups around people doing their own AI, then Huggingface comes to mind: https://nitter.net/huggingface I also recall them at least discussing to implement GPT3, but they were aware of the problem that it needs quite some powerful servers. There's generally a lot going on on Twitter and Reddit, it's just necessary to focus one that topic and not getting pulled into politics or other distractions... In my case, I first have to learn the math, otherwise it's pointless. Maybe I'm getting the Brilliant app soon, but first I'll try some other sites and Youtube channels. Also I want to move and buying a new PC after that, so that I won't have to move it.
>>5519 I imagine some months down the line I might be able to write an SDF importer for Godot. That way simulation training done in Bullet can be used in games and demos easily. >>5536 Yeah, Huggingface provides a lot of useful tools and models for tokenizing text and stuff. The models used in TalkToWaifu were produced by Huggingface. They're definitely leading in open-sourcing NLP and making it available for everyone.
related xpost for simulators >>5667
This guy here makes interesting stuff with voxels: https://nitter.dark.fail/tuxedolabs - Very realistic environments. Youtube video: https://youtu.be/aAgVSTrqNOc - Not directly slicof life, but if one wants to learn about simulators and how to train AI in such environments, this might be useful.
>>8486 Wow, thanks Anon! That's pretty inspiring actually. I'll make time to dig through this guy's work and see if I can figure out what he's doing. Also, thank you for using nitter instead.
>>8491 You're welcome. Teardown seems to be the name of the Game now, but there's more: >voxel engine /plugin The guy reporting on it, has more videos: https://youtu.be/Jnghq0yTmZo >"Physics" engine /plugin Primary liquids, not sure how realistic. https://youtu.be/DBcqiQLp7lY What we would need is rather a household with people, and all kinds of things, where the behavior depends on the material, temperature, and so on. His vids are more aimed towards gaming and fun or art. At the end of the physics video he points out that more is possible.
>>8501 I like the idea of creating a robowaifu simulator using voxels. I assume they are a pretty efficient way to create interactive physics, just from looking at the play from that vidya.
https://www.raylib.com/ might be interesting for simulators or virtual waifus. It's for gaming and works basically everywhere. Its very modular, modules can be used on their own.
>>8925 Thanks for the reminder Anon. Some other anons mentioned it here before, but I'd gotten so busy I'd forgotten about. You've reminded me about it now. Here's a cleaned-up version of his animation example code, looks pretty simple to use: /******************************************************************************* * * raylib [models] example - Load 3d model with animations and play them * * This example has been created using raylib 2.5 (www.raylib.com) * raylib is licensed under an unmodified zlib/libpng license (View raylib.h * for details) * * Example contributed by Culacant (@culacant) and reviewed by Ramon * Santamaria (@raysan5) * * Copyright (c) 2019 Culacant (@culacant) and Ramon Santamaria (@raysan5) * ******************************************************************************* * * To export a model from blender, make sure it is not posed, the vertices need *to be in the same position as they would be in edit mode. and that the scale *of your models is set to 0. Scaling can be done from the export menu. * ******************************************************************************/ #include "raylib.h" #include <stdlib.h> int main(void) { // Initialization //---------------------------------------------------------------------------- const int screenWidth = 800; const int screenHeight = 450; InitWindow(screenWidth, screenHeight, "raylib [models] example - model animation"); // Define the camera to look into our 3d world Camera camera = {0}; camera.position = (Vector3){10.0f, 10.0f, 10.0f}; // Camera position camera.target = (Vector3){0.0f, 0.0f, 0.0f}; // Camera looking at point // Camera up vector (rotation towards target) camera.up = (Vector3){0.0f, 1.0f, 0.0f}; camera.fovy = 45.0f; // Camera field-of-view Y camera.type = CAMERA_PERSPECTIVE; // Camera mode type // Load the animated model mesh and basic data Model model = LoadModel("resources/guy/guy.iqm"); // Load model texture and set material Texture2D texture = LoadTexture("resources/guy/guytex.png"); // Set model material map texture SetMaterialTexture(&model.materials[0], MAP_DIFFUSE, texture); Vector3 position = {0.0f, 0.0f, 0.0f}; // Set model position // Load animation data int animsCount = 0; ModelAnimation* anims = LoadModelAnimations("resources/guy/guyanim.iqm", &animsCount); int animFrameCounter = 0; SetCameraMode(camera, CAMERA_FREE); // Set free camera mode SetTargetFPS(60); // Set our game to run at 60 frames-per-second //---------------------------------------------------------------------------- // Main game loop while (! WindowShouldClose()) // Detect window close button or ESC key { // Update //-------------------------------------------------------------------------- UpdateCamera(&camera); // Play animation when spacebar is held down if (IsKeyDown(KEY_SPACE)) { animFrameCounter++; UpdateModelAnimation(model, anims[0], animFrameCounter); if (animFrameCounter >= anims[0].frameCount) animFrameCounter = 0; } //-------------------------------------------------------------------------- // Draw //-------------------------------------------------------------------------- BeginDrawing(); ClearBackground(RAYWHITE); BeginMode3D(camera); DrawModelEx(model, position, (Vector3){1.0f, 0.0f, 0.0f}, -90.0f, (Vector3){1.0f, 1.0f, 1.0f}, WHITE); for (int i = 0; i < model.boneCount; i++) { DrawCube(anims[0].framePoses[animFrameCounter][i].translation, 0.2f, 0.2f, 0.2f, RED); } DrawGrid(10, 1.0f); // Draw a grid EndMode3D(); DrawText("PRESS SPACE to PLAY MODEL ANIMATION", 10, 10, 20, MAROON); DrawText("foo_bar_baz", screenWidth - 200, screenHeight - 20, 10, GRAY); EndDrawing(); //-------------------------------------------------------------------------- } // De-Initialization //---------------------------------------------------------------------------- UnloadTexture(texture); // Unload texture // Unload model animations data for (int i = 0; i < animsCount; i++) { UnloadModelAnimation(anims[i]); } RL_FREE(anims); UnloadModel(model); // Unload model CloseWindow(); // Close window and OpenGL context //---------------------------------------------------------------------------- return 0; } https://www.raylib.com/examples.html
>>8926 Ah, didn't look for it. Here: >>5810 >>2018
I am currently building a framework for 2D simulations, similar to dwarf fortress. The goal is to have a simulation that is as realistic as possible, while also using up as little performance as reasonable. I want to train multi-agents with evolutionary model optimization in the simulation; to test a few ideas on how to solve artificial general intelligence without having to bother with RobotVision and ArtificalMuscleMovement. Main goal is to solve general AI but I also want to try and solve NaturalLanguage using a new (?) approach. Going to be open source one day. (it's not my day-job to code this so progress is kinda slow. Anyone interested in it?)
>>9032 >Anyone interested in it? Sure ofc, Anon. We're always interested in that kind of thing here. Give us more details please.
>>9032 When I first got started with neural networks twenty years ago I made a simple evolution simulation in Game Maker with prey and predators. I've always wondered if they were given the ability to make different sounds and hear each other walking if they would've formed their own crude language eventually. I think that would be a fun project. I also want to solve general AI and feel that AI needs a more complex environment to really learn and grow. Most AI research focuses on maxing something out but in real life there's always something more to learn.
>>9034 >When I first got started with neural networks twenty years ago Wow that's pretty remarkable I had no idea anyone here had that kind of experience. It would be really nice if you conducted a beginner's class on these topics if you're up for something like that Anon. Cheers.
>>9045 Nah, I just played around with them for a bit as a kid after watching Chobits. I didn't really get into AI until around 2015. I could probably still teach things but there's lots of material for that on the web and I'd rather work on my own stuff.
>>9047 Oh haha I see. Well, it's still good to know you're here. Good luck with your 'stuff' Anon. I too found Chobits inspirational when I was young.
Open file (281.77 KB 1441x768 physx.jpg)
Has anyone tried PhysX? The most interesting feature of it for me is the GPU acceleration because PyBullet isn't really ideal for training thousands of iterations simultaneously. https://www.youtube.com/watch?v=K1rotbzekf0 https://github.com/petrikvladimir/pyphysx I'd like to start moving on from just learning about AI and actually apply it to some real problems that can be transferred to the real world. My goal is to start assembling my own robowaifu by the end of the year that can do some basic task like locate my hand and give me a high five or something.
>>9708 Yes, I've played with it bit (cloth sims & rigid body collisions mostly). Nvidia is plainly an evil Big Tech/Gov entity like the others, but they have made a remarkable push to make their GPU ecosystem easy to use if you at least have the basics of C programming under your belt.
While I wait for stuff to train I started working a side-project for playing video games with hindsight experience replay and became fascinated with the idea of repurposing the model afterwards for a piano simulation, similar to how they trained a robot hand to solve a Rubik's cube in a simulation then did transfer learning to the real world. A ton of progress has been made in making computer vision and reinforcement learning more efficient. Just two years ago it was probably too early to make any useful simulation and reasonably train an agent in it to do something interesting but now I think it's within reach. Maybe it's only an exciting idea to me but I think it would be a great way to demonstrate how far along AI has progressed and that anyone can make their own waifu play piano MIDI files given. We could demo other capabilities with it too like making her say the title of the song before playing and generate some virtual waifu hype. The simulation code would also provide a base for others to get started doing other custom simulations such as moving arms, balancing, and walking.
>>10073 >Maybe it's only an exciting idea to me No, I think it's a great idea Anon. I agree this would both be an innovative and useful way to build up training for AIs/Robowaifus.
Over the past few days I've gotten the latest Blender and UE4 working on my machines.
Someone mentioned Webots: Robot Simulator here >>10531 Link: https://cyberbotics.com/ >It has been designed for a professional use, and it is widely used in industry, education and research. Cyberbotics Ltd. maintains Webots as its main product continuously since 1998. >Webots core is based on the combination of a modern GUI (Qt), a physics engine (ODE fork) and an OpenGL 3.3 rendering engine (wren). It runs on Windows, Linux and macOS. Webots simulations can be exported as movies, interactive HTML scenes or animations or even be streamed to any web browser using webgl and websockets. >Robot may be programmed in C, C++, Python, Java, MATLAB or ROS with a simple API covering all the basic robotics needs.
>>1708 I apologize Anon, for not thanking you for this image when you posted it. I actually did appreciate it back then, but I was too distracted to share my gratitude at the time. So thanks! :^)
>>10073 We'd be interested to hear how both the training and your 'side-project' worked out Anon. Any news to share with us?
Open file (1.31 MB 2400x3600 Roastinator.png)
>>1092 >>1093 >>1101 Already been meme'd. I think something very similar to this was originally proposed as a "troll" on a certain chan, which literally led to some articles and "focus groups" that led to the creation of a monitoring group who has eyes on all "robot" threads/convos across the internet. > The roastie fears the robot-samurai.
Open file (56.08 KB 500x356 beewatcher_lg.gif)
>>11010 Haha, well that's interesting Anon! Can you give us more details on these 'watchers'? Somebody might need to keep eyes on them tbh. >And now all the Hawtchers who live in Hawtch-Hawtch are watching on watch watcher watchering watch , watch watching the watcher who's watching that bee. You're not a Hawtch-Watcher you're lucky you see!” >t― Dr . Seuss , Did I Ever Tell You How Lucky You Are?
>>11013 Hmmm. Use machines to keep an eye on machines? In a time of deepfake video and digitally altered footage, I just hope they believe the camera feeds they're watching. 😉
What looks to be a very useful header-only C++ wrapper around the OpenGL C API. I'll try to make some time to have a look at it over the summer. https://oglplus.org/oglplus/html/index.html https://github.com/matus-chochlik/oglplus
Open file (407.22 KB 892x576 MoxRigForBlender.png)
Open file (23.94 KB 566x698 momo_rig.png)
Here's something I found through our Japanese colleagues. MoxRig / MomoRig https://mox-motion.com/ - I didn't try it out, but it seems to be useful for animation of human-like movement and simulation of robots.
>>11497 Neat! Thanks Anon, I'll give it a lookover.
>>11022 >emoji friendly islamic reminder this is a chan don't use emoji please

Report/Delete/Moderation Forms
Delete
Report

no cookies?