So I decided to try to update part of my toolchain...
-
@Gąska said in So I decided to try to update part of my toolchain...:
See above
Which bit? Magic to be determined later or every text editor will need to have a file picker widget?
-
@Jaloopa said in So I decided to try to update part of my toolchain...:
@Gąska said in So I decided to try to update part of my toolchain...:
See above
Which bit?
The one where I talked about exactly what you asked.
Magic to be determined later or every text editor will need to have a file picker widget?
You think this topic has only one page? You need to look a bit further back.
-
@Gąska said in So I decided to try to update part of my toolchain...:
You think this topic has only one page?
This topic has many pages. I think the maximum page size is 50…
-
@Jaloopa said in So I decided to try to update part of my toolchain...:
@Gąska said in So I decided to try to update part of my toolchain...:
See above
Which bit? Magic to be determined later or every text editor will need to have a file picker widget?
I believe that the file will have other tags associated with it in place of the subdirectories of the path. So I'd guess you'd supply an array / list of those along with the filename.
But it's also possible that I'm confusing what @Gąska said with what someone else said at this point.
-
@boomzilla said in So I decided to try to update part of my toolchain...:
@Jaloopa said in So I decided to try to update part of my toolchain...:
@Gąska said in So I decided to try to update part of my toolchain...:
See above
Which bit? Magic to be determined later or every text editor will need to have a file picker widget?
I believe that the file will have other tags associated with it in place of the subdirectories of the path. So I'd guess you'd supply an array / list of those along with the filename.
But it's also possible that I'm confusing what @Gąska said with what someone else said at this point.
Which may not be unambiguous (there could be many files with the same set of tags/file name but different GUID). And is just a path by another name, minus the path separators.
-
@Benjamin-Hall said in So I decided to try to update part of my toolchain...:
@boomzilla said in So I decided to try to update part of my toolchain...:
@Jaloopa said in So I decided to try to update part of my toolchain...:
@Gąska said in So I decided to try to update part of my toolchain...:
See above
Which bit? Magic to be determined later or every text editor will need to have a file picker widget?
I believe that the file will have other tags associated with it in place of the subdirectories of the path. So I'd guess you'd supply an array / list of those along with the filename.
But it's also possible that I'm confusing what @Gąska said with what someone else said at this point.
Which may not be unambiguous (there could be many files with the same set of tags/file name but different GUID). And is just a path by another name, minus the path separators.
That sounds familiar.
-
@dkf said in So I decided to try to update part of my toolchain...:
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@levicki said in So I decided to try to update part of my toolchain...:
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@Gąska said in So I decided to try to update part of my toolchain...:
taking care of converting user input into file IDs/handles.
Sure, sure, but.... How would it be done?
Again, I say, MAGIC!
Are we now onto MFFS?
I like it! We need to patent that, stat!
I'd have thought that you'd go for TSFS.
BTDT. It had non determinism baked in as a feature.
-
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@dkf said in So I decided to try to update part of my toolchain...:
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@levicki said in So I decided to try to update part of my toolchain...:
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@Gąska said in So I decided to try to update part of my toolchain...:
taking care of converting user input into file IDs/handles.
Sure, sure, but.... How would it be done?
Again, I say, MAGIC!
Are we now onto MFFS?
I like it! We need to patent that, stat!
I'd have thought that you'd go for TSFS.
BTDT. It had non determinism baked in as a feature.
Oh wait! That has to be the acronym for the file system. Now we need to backronym it...
-
@dcon said in So I decided to try to update part of my toolchain...:
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@dkf said in So I decided to try to update part of my toolchain...:
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@levicki said in So I decided to try to update part of my toolchain...:
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@Gąska said in So I decided to try to update part of my toolchain...:
taking care of converting user input into file IDs/handles.
Sure, sure, but.... How would it be done?
Again, I say, MAGIC!
Are we now onto MFFS?
I like it! We need to patent that, stat!
I'd have thought that you'd go for TSFS.
BTDT. It had non determinism baked in as a feature.
Oh wait! That has to be the acronym for the file system. Now we need to backronym it...
Binary-Tabled Distribution Tap Filesystem.
-
If your ideal filesystem is basically flat, there's a name that's obvious: SSFS, aka the SpectateSwamp File System.
-
@pie_flavor said in So I decided to try to update part of my toolchain...:
C-average Indians
Three stars?
-
@levicki said in So I decided to try to update part of my toolchain...:
What's wrong with flat filesystem?
If you only have a few tens of files, nothing.
If you've got hundreds of thousands of them, you might want a bit more organisation…
-
@levicki said in So I decided to try to update part of my toolchain...:
Filesystem should be flat, database should be organized and indexed.
like some kind of Allocation Table for the Files? Or like a Master File Table or something?
-
@levicki said in So I decided to try to update part of my toolchain...:
fragmentation
And having a flat file system is going to help that how? (I have close to 1,000,000 files on this system - that's after cleaning up multiple build outputs.)
-
@levicki said in So I decided to try to update part of my toolchain...:
Because you wouldn't mix the hierarchy and metadata with content?
Hmm. Last time I ran a defragger, I seem to remember the directories being all clustered together. So current file systems aren't mixing hierarchy and content at the sector level now.
edit: Not to say they don't due to fragmentation.
-
@pie_flavor Which part of what I said does not apply? For your convenience, I'll quote myself.
I guess my take on this is that:
Yeah, maybe file names suck and should be "replaced".
But at the end of the day, the user will need to interact with a file, and unless the file is completely hidden behind the application (including serialization and transfer to peers, etc), the file will need some kind of user-readable identifier. This identifier is a definite descriptor (in the sense of Russell), so arguably qualifies as a "name". So we're back to square one.
Pretty sure developers have to interact with files. Even if it's just to get the file into the IDE.
Pretty sure a database query or handler that returns a unique result is a definite description. By definition.
The reality is that this is the situation we're in today, too. File systems keep evolving internally, pushing more metadata in there to work more robustly. We still need a "name", and filenames are good enough.
Still true whether you're a user or developer.
But filenames do suck, and should be replaced. I had a lot of hope for MacOS's Spotlight thing way back when. Maybe AI can help us figure out some sensible universal tagging system. There just have to be better ideas for how we interact with files.
Still true.
-
Interesting. Looks like the move to 2019 will be a lot easier.
C++ Binary Compatibility and Pain-Free Upgrades to Visual Studio 2019 - C++ Team Blog
Visual Studio 2019 pushes the boundaries of individual and team productivity. We hope that you will find these new capabilities compelling and start your upgrade to Visual Studio 2019 soon. As you are considering this upgrade, rest assured that Visual Studio 2019 makes it distinctively easy to move...
-
@levicki said in So I decided to try to update part of my toolchain...:
Because defragmenting a file would be as easy as moving it to the start of free space?
Uh, it is this already, almost regardless of what filesystem you use....
-
@levicki said in So I decided to try to update part of my toolchain...:
@dcon said in So I decided to try to update part of my toolchain...:
And having a flat file system is going to help that how?
Because defragmenting a file would be as easy as moving it to the start of free space? Because you wouldn't mix the hierarchy and metadata with content? Because filesystem we discussed would require a length hint upon file creation to optimize storage?
You do seem to skip over the problem of finding a free space big enough to store your defragmented file, and maybe having to shuffle other files a bit so that the gaps between them don't waste as much space.
NTFS always tracks gap sizes using its $Bitmap file so it immediately knows what is free. And if it has to move files, the MFT is a "flat" structure which points to everything taking space on the drive because the files which directories point to are also in the MFT.
-
@levicki said in So I decided to try to update part of my toolchain...:
Why do you people make authoritative claims about stuff you are clueless about?
Welcome to the forums!
-
@levicki said in So I decided to try to update part of my toolchain...:
Why do you people make authoritative claims about stuff you are clueless about?
NO U
-
@levicki said in So I decided to try to update part of my toolchain...:
Why do you people
What do you mean "you people"? At least one forum member identifies as an attack helicopter.
-
@Gribnit
Just because we ran out of foxes ...
-
@Luhmann said in So I decided to try to update part of my toolchain...:
@Gribnit
Just because we ran out of foxes ...Thanks for reminding me to spend some time pissing people off.
-
@levicki said in So I decided to try to update part of my toolchain...:
@JBert said in So I decided to try to update part of my toolchain...:> > You do seem to skip over the problem of finding a free space big enough to store your defragmented file, and maybe having to shuffle other files a bit so that the gaps between them don't waste as much space.
Free space big enough is always at the end unless you just ran out of space.
If there's "virgin" space enough then files are unlikely to have been fragmented in the first place. However, if the disk is getting a lot of use it is possible that there are a whole bunch of clusters which are free but they might not be next to each other at all. So free space is hardly ever at the end until it's a virgin disk or the system is aggressively moving files to the front of the disk, but you never went into detail about the latter.
@JBert said in So I decided to try to update part of my toolchain...:
NTFS always tracks gap sizes using its $Bitmap file so it immediately knows what is free. And if it has to move files, the MFT is a "flat" structure which points to everything taking space on the drive because the files which directories point to are also in the MFT.
I provided a quote above which says that NTFS uses B-trees and stores parts of large directories in clusters which are external to MFT so no, MFT is not flat nor it is enough to read just MFT to find stuff.
That quote talks of how directories are implemented. If you happen to have quoted it from http://www.ntfs.com/ntfs-mft.htm (you didn't specify the source) then there's also this a bit higher up:
The master file table allocates a certain amount of space for each file record.
In other words, the MFT itself is a flat array of records*, with each record having a fixed size and saying on what areas of the disk the actual file content or directory listing is stored (mind you, a directory is only an "index file" describing what other files are contained within it and each index entry - whether it's stored in a B-tree structure or not - points back to a different MFT entry for that file in question).
So, I thought we were talking about defragmenting. For defragmenting in NTFS you only need to iterate over the flat MFT structure to find all files and their location on disk (because again, the MFT is the primary source of truth about files and directories), and moving the chunks of data while correcting the MFT entry of the file directly afterward will keep everything working nicely. At this point I don't see how it matters in what directories a file is stored.
@levicki said in So I decided to try to update part of my toolchain...:
Why do you people make authoritative claims about stuff you are clueless about?
I did some reading but I never claimed I'm the pope of file systems. Also, I'm not the one saying that we really, really need a different filesystem. I would expect that anyone wanting to come up with a new filesystem should first learn what the actual internals and drawbacks are of existing ones.
* One thing I was mistaken about is that I thought the MFT is of a fixed size determined when formatting your disk. Apparently it can grow to accommodate use cases where you have millions of small files, that way you don't run out of MFT entries before every cluster of your hard disk is filled. Normally the MFT always resides within a single continuous region of your disk (Windows reserves ~12%), but if it grows more it will try to use free clusters all over the place and thus will be spread out over several pieces of disk. To keep track of where on disk the MFT is stored there is actually an entry in the MFT reserved for the MFT itself.
-
@Gribnit said in So I decided to try to update part of my toolchain...:
@levicki said in So I decided to try to update part of my toolchain...:
Why do you people
What do you mean "you people"? At least one forum member identifies as an attack helicopter.
And another as a kaiju.
-
Hey guys, I found that system that was mentioned that uses UIDs for files instead of paths!
Turns out it can't find its own files either...
Edit: Thank $deity they don't use jargon like "cursor" in this OS...
-
@Tsaukpaetra It's the best programming environment ever devised. @blakeyrat says so, so it must be true.
-
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
this OS...
Windows 10 including Maps in the OS by default is way behind the times. Apple did it literally forever ago!
It's even in the Control Panel (like any sensible mapping program would be!
-
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
It's even in the Control Panel (like any sensible mapping program would be!
It is an Apple mapping program, what else do you expect?
-
@Groaner said in So I decided to try to update part of my toolchain...:
Looks like the move to 2019 will be a lot easier.
`:admiralakbar:`
-
@boomzilla said in So I decided to try to update part of my toolchain...:
@Groaner said in So I decided to try to update part of my toolchain...:
Looks like the move to 2019 will be a lot easier.
`:admiralakbar:`
Oh, that's not something I plan on jumping on right away, anyway. I have a backlog to get back to. And even if I have to rebuild everything again next time around, it'll be easier since now I know that some of my dependencies haven't been updated since 2013, so there's no need to go look for updated versions. In fact, for some of them, the versions I have locally may be the most up-to-date versions on the planet as they actually compile and run with latest Ogre.
-
@Groaner said in So I decided to try to update part of my toolchain...:
, the versions I have locally may be the most up-to-date versions on the planet as they actually compile and run with latest Ogre.
PRs on dead projects accepted?
-
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@Groaner said in So I decided to try to update part of my toolchain...:
, the versions I have locally may be the most up-to-date versions on the planet as they actually compile and run with latest Ogre.
PRs on dead projects accepted?
Possibly, if I can find the relevant repos. I know that a couple of the "dead" plugins abandoned by their authors were taken on by the community for support. Then there's stuff like RakNet, which Oculus and the community at large have done almost zero with since it went open source a few years ago.
-
@Groaner said in So I decided to try to update part of my toolchain...:
RakNet
*googles...* I wonder what the benefit of that is over the game engine's built-in networking?
As it stands, we've had to hack around the provided online subsystems for all our supported platforms anyways do...
-
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@Groaner said in So I decided to try to update part of my toolchain...:
RakNet
*googles...* I wonder what the benefit of that is over the game engine's built-in networking?
As it stands, we've had to hack around the provided online subsystems for all our supported platforms anyways do...
Depends on the game engine. As I had no networking component, I started out trying to roll something with Boost.Asio and then after much suffering I decided to give RakNet a shot, which solved pretty much all my problems. At the time, RakNet was closed-source and you had to pay up if you had more than $100k of revenue, which is why I was hesitant to move to it. Shortly after I did, it went open source.
In addition to the core features like entity replication, remote procedure calls, and different reliability channels, it also has a lot of nice plugins like a lobby server, matchmaking server, some barebones mail and web servers if I remember correctly, and a client-side prediction module for Ogre, which basically put all that suffering I was talking about to rest. My understanding is that the stock Unity netcode is also based on older RakNet (I use 4.x in my project).
Now that pretty much everyone is on Unreal or Unity, the need for these separate components is diminished. Kevin gives sort of a history of how the project (and the whole industry, for that matter) changed.
-
@Groaner said in So I decided to try to update part of my toolchain...:
As I had no networking component
Ah, yeah. So that's the use case then. If you're not using a mainstream game engine or are otherwise hand-rolling communications.
Makes me wonder how robust our hand-rolled control communications are on the actual Internet, if the game servers ever ended up needing to cross out to the Internet for control sockets...
-
@levicki said in So I decided to try to update part of my toolchain...:
@Gąska Just stop that
bullshit please.I will reiterate the important takeaways from the discussion:
- Proper way to install C++ runtime is by running installer and putting them in system folder where they belong to be used by every application.
- DLLs are good, save resources and should be used to avoid binary code duplication.
- If anything was WTF in my story it was the requirement to have target OS which is not updated since release and then request software which needs sha256 signed files.
As for binary patching, I even have no clue why you brought it up in the first place and frankly I don't want to know.
Self-righteous anger I see. To the forums welcome, I say. Right in you fit, it seems. Fun we will have, I hope.
-
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@Groaner said in So I decided to try to update part of my toolchain...:
RakNet
*googles...* I wonder what the benefit of that is over the game engine's built-in networking?
The main benefit was when your engine didn't have built-in networking. And on Unity, RakNet was your built-in networking (not sure about now - maybe they've changed it, maybe they haven't). Other than that, I guess usual tradeoffs apply - some workflows are supported better, some are supported less, and you have to try it out yourself to see if it's worth it in your particular case because no one will tell you the full truth.
-
@Gąska said in So I decided to try to update part of my toolchain...:
no one will tell you the full truth
Because no-one knows it; there's too many weird variables for people's individual situations.
-
@dkf said in So I decided to try to update part of my toolchain...:
@Gąska said in So I decided to try to update part of my toolchain...:
no one will tell you the full truth
Because no-one knows it
That too. But this doesn't really make your situation any better.
-
@Gąska said in So I decided to try to update part of my toolchain...:
But this doesn't really make your situation any better.
It stops people from running around trying to find a guru who knows THE TRUTH and instead finding a charlatan selling snake oil. I call that a win.
-
@dkf I meant, that no one knows full truth isn't any better than if they knew but refused to tell. And people will still chase gurus alright - because even though no one knows everything, there are still people who claim they do.
-
@Gąska said in So I decided to try to update part of my toolchain...:
@Tsaukpaetra said in So I decided to try to update part of my toolchain...:
@Groaner said in So I decided to try to update part of my toolchain...:
RakNet
*googles...* I wonder what the benefit of that is over the game engine's built-in networking?
The main benefit was when your engine didn't have built-in networking. And on Unity, RakNet was your built-in networking (not sure about now - maybe they've changed it, maybe they haven't). Other than that, I guess usual tradeoffs apply - some workflows are supported better, some are supported less, and you have to try it out yourself to see if it's worth it in your particular case because no one will tell you the full truth.
Back when the company was working on a Unity project, I kept hearing about something called Photon, which was a plugin to supplant the stock RakNet implementation, but I didn't pay too much attention as I made only a couple contributions to that project and I have gone on record many times here about how much I despise Unity.
-
Maybe it's time to take another crack at this. The recency of activity is very promising!
Time to fire up CMake and prepare for more suffering. First thing that comes up is an error about needing a Qt path. Okay, so let's download the latest Qt, and skip all the annoying pages about making sure to contribute back to the community if you have an open source license, blah blah.
I just love an online installer that can't figure out what it's supposed to install. Okay, let's follow the links on the site for an offline installer...
Fucking hell. That might explain why the online installer shit itself. Looks like getting a working copy of Qt will be difficult.
Wait a minute, maybe I have a compatible version of Qt lying around from a previous install. Let's check the HDD:
Success!
Double success! Well, it's a success in that it's no longer bitching about not being able to find a Qt and instead bitching about not finding Ogre. Looks like I need to download 1.12. That's heartening because upgrading my project to 2.x would be a nightmare, and incremental upgrades with Ogre are usually pretty smooth. 1.12 is only needed to build Ogitor at this point, and if I need to update my own codebase, that's still on 1.11 from last time. Let's see what the future brings....
-
I got Ogre 1.12 built, trying x64 this time since the only Qt package I have for VS2017 seems to be x64. If this endeavor works, it might be time to bring the rest of my project into x64 land, even though memory pressure has not been a concern yet.
Either way, with 1.12 built, it's time to go back to Cmake and try to plug it in to Ogitor:
Apparently, version 1.12 is incompatible with version 1.12. I wonder if it silently requires x86?
On the bright side, the Qt download site seems to be operational again.
-
@Groaner Does it want you to specify 1.12.10 instead of 1.12?
-
@Unperverted-Vixen said in So I decided to try to update part of my toolchain...:
@Groaner Does it want you to specify 1.12.10 instead of 1.12?
I don't think so. I ended up rebuilding everything as x86 and that issue disappeared.
So... a couple weeks later, after many trials and tribulations that involved:
- rebuilding Ogre with various options (Ogitor requires the new Ogre - Qt integration, and if you hadn't checked that box in Cmake, you would get
unresolved external symbol MyClass::qt_metacalllinker errors due to the project files not being set up properly and the moc files not being generated) - upgrading to latest Cmake (since some versions of Boost are "too new" to be supported)
- manually entering dependency paths to every project in a 30+ project VS solution
I have this!
Not 100% yet, but the furthest I've gotten trying to update this. I imagine I'll need to wire up some components and plugins to get it fully functional, but it's very nice having something that compiles and runs.
In all my digging, I came across a discussion of how and why the Ogre ecosystem is dying while the masses have moved over to the Unity/Unreal duopoly if they aren't waiting for Godot. I have thus far resisted migrating as I hate Unity, and while Unreal is probably a better fit for my project of the two options, or the lesser of two evils, from what I've heard from you guys, there will probably be some pain in going down that route as well.
There are so many systems involved that porting to any other tech would be a major effort and probably leave development dead in the water for a year or two. Maybe it's the least bad option to stay on an aging stack that is losing market share but is at least mostly still supported and maintained?
Working on such a massive project for so long sometimes makes me feel like the scientist in Stripperella who emerges from the basement after 30 years, having invented a cellular phone with a rotary dial that only weighs 10 pounds. A lot has changed in the industry in the last ~10 years, not just in terms of what tech is hot (that change is more or less a constant), but the zeitgeist and society at large. I'm not making a Skyrim, but while a Skyrim would be a smash hit in 2011, you can't make a Skyrim in 2021. And that's not just because audiences have tired of Skyrims, but they have come to expect things of their Skyrims that were never and could never be in the circa 2011 design documents. Developers who plan on ambitious 5+ year projects, heed this warning, as the world in which you eventually deliver your project may be a very different one from the one in which you designed it.
- rebuilding Ogre with various options (Ogitor requires the new Ogre - Qt integration, and if you hadn't checked that box in Cmake, you would get
-
Awww yeah.
The above black screen had stymied me for a bit until I figured out where the app was dumping its Ogre log files (which are an invaluable feature of any Ogre application) -
C:\Users\[me]\Documents\Ogitor- the user's home directory. How very *nix!Said log file suggested to me that this black screen was caused by texture assets failing to load on account of no image codec having been specified. I had forgotten that in more recent versions of Ogre, you have to pick which image codec to use rather than it Just Working™ out of the box. The burden of choice! How very *nix! Either way, one more line added to
plugins.cfgto load the image codec, and we're on our way.There are still some other things I need to address, mainly SkyX and HydraX support. The above scene is just using a skybox so it loads fine, but my project heavily uses said plugins as they were stunning by 2009 standards and still look reasonably ok today, even though they haven't really been updated in 5+ years.
https://www.youtube.com/watch?v=FL02s_MGIQc
If I can't get them working, it's not a huge deal as I can inject the parameters for my scene elsewhere, but it would be really nice to have it all under one roof.
-
Looks like I already have the code for the HydraX/SkyX editor modules, but I do not have the project files for them. Of course, modifying
CMakeLists.txtto create projects for them and running Cmake again obliterates all my manual edits to correct include and lib file paths. Yes, I learned this the hard way!Since I'm sick of doing these manual corrections, I now have a dilemma: do I try to find and correct the include/lib path code in the Cmakefiles so that it emits correct paths, submit a pull request and potentially make this easier for the next human being in my situation, or do I just make a simple utility or even a Powershell script that does a string find/replace on the
.vcxprojfiles? I do not know Cmake (well, I know enough to be dangerous), but I do know all about find/replace on XML files. It's a clbuttic problem with many easy and viable solutions!
