Dolt
-
I came across this:
https://docs.dolthub.com/introduction/why-doltIt's actually fascinating. It combines MySQL and git. Which surely earns it a place in this category.
The ability to see the data as it was at any point in history is, of course, super valuable and I would love to have this. Being MySQL, I never will.
-
@boomzilla said in Dolt:
It combines MySQL and git. Which surely earns it a place in this category.
In particular we aim to implement a superset of the MySQL dialect standard by standing up a MySQL compatible server written in Go.
Yes, definitely earned its place.
@boomzilla said in Dolt:
The ability to see the data as it was at any point in history is, of course, super valuable and I would love to have this.
Like Oracle's Flashback, but without the limitation of relying on undo logs. Seems potentially handy.
-
@boomzilla That's an unfortunate name, when sans-serif type makes it look like a synonym for "idiot".
edit: The concept of it would have solved a design dilemma this morning, actually.
-
@Bim-Zively ah, okay, this is the first I'm noticing that being uppercase vs an L. The corrected name is much worse.
Ah, okay, my freaking eyeballs got successfully Overtoned, I'll get ye...
-
@Bim-Zively said in Dolt:
@boomzilla That's an unfortunate name, when sans-serif type makes it look like a synonym for "idiot".
edit: The concept of it would have solved a design dilemma this morning, actually.
It's not the font. It's the actual name:
But to be fair, one of the programs it emulates is called "git."
-
@boomzilla said in Dolt:
@Bim-Zively said in Dolt:
@boomzilla That's an unfortunate name, when sans-serif type makes it look like a synonym for "idiot".
edit: The concept of it would have solved a design dilemma this morning, actually.
It's not the font. It's the actual name:
But to be fair, one of the programs it emulates is called "git."
Yeah that justified it for me. I like a dumb data layer. Data at rest getting fetful is not a comforting notion.
-
This post is deleted!
-
I've always wondered what would happen if we combined csv files with git. I fully expected to see it at my current workplace first but now I see someone has made a business model out of it. Well done.
-
My biggest question here would be "is this even remotely performant?"
-
@sloosecannon said in Dolt:
My biggest question here would be "is this even remotely performant?"
So their goal is:
As stated at the outset, our goal is to get Dolt to the point where it is no more than 2-4 times slower than MySQL for sysbench's standard test suite of OLTP and non-OLTP tests.
They aren't even there yet.
-
@sloosecannon said in Dolt:
My biggest question here would be "is this even remotely performant?"
Mine is how much storage does it use? If permanently preserves every modification to a large, rapidly-changing database, that could chew through storage like mad.
-
@HardwareGeek said in Dolt:
@sloosecannon said in Dolt:
My biggest question here would be "is this even remotely performant?"
Mine is how much storage does it use? If permanently preserves every modification to a large, rapidly-changing database, that could chew through storage like mad.
It'll absolutely do that if the records change often.
If being able to retrieve the state of a record at any point in time is a requirement then however that's implemented will do the same.
At scale storage is (relatively) cheap though.
-
@loopback0 said in Dolt:
If being able to retrieve the state of a record at any point in time is a requirement
Is there any real system where this is a requirement?
-
@HardwareGeek said in Dolt:
@loopback0 said in Dolt:
If being able to retrieve the state of a record at any point in time is a requirement
Is there any real system where this is a requirement?
Filed under: SSDS
-
@HardwareGeek said in Dolt:
@loopback0 said in Dolt:
If being able to retrieve the state of a record at any point in time is a requirement
Is there any real system where this is a requirement?
I'm sure someone will come up with one.
-
@HardwareGeek said in Dolt:
@loopback0 said in Dolt:
If being able to retrieve the state of a record at any point in time is a requirement
Is there any real system where this is a requirement?
Filed under: SSDS
That's an interesting definition of real you have there.
-
So why not use a traditional database which supports setting up an audit table?
-
@HardwareGeek said in Dolt:
@HardwareGeek said in Dolt:
@loopback0 said in Dolt:
If being able to retrieve the state of a record at any point in time is a requirement
Is there any real system where this is a requirement?
Filed under: SSDS
That's an interesting definition of real you have there.
It pretty much exists. Seems fair.
-
So why not use a traditional database which supports setting up an audit table?
I can only say "no reason" after mocking you for an indefinite length of time, unless maybe it is horizontally distributable better than some SQL db. Unfortunately, I have not received the latest shipment of WebScale and am running low on anything topical. Yo mama. Yo mama. Yo mama. Yo mama. Please extend as necessary.
JCR has had versioning forever, BTW. Over some kind of tar file abomination for awhile.
If many of your users travel at relativistic speed, the use case seems much stronger.
-
@loopback0 said in Dolt:
@sloosecannon said in Dolt:
My biggest question here would be "is this even remotely performant?"
So their goal is:
As stated at the outset, our goal is to get Dolt to the point where it is no more than 2-4 times slower than MySQL for sysbench's standard test suite of OLTP and non-OLTP tests.
They aren't even there yet.
Not too hot at maths either. 19.5 is neither the geometric (11.7) nor even the arithmetic (16.3) mean of those multiples
-
@kazitor standard is geothmetic meandian.
-
@boomzilla said in Dolt:
But to be fair, one of the programs it emulates is called "git."
"git" as in
?
-
If your data is in the CloudTM, there is always Snowflake
-
@BernieTheBernie said in Dolt:
@boomzilla said in Dolt:
But to be fair, one of the programs it emulates is called "git."
"git" as in
?Aye. Git in the van!
-
-
-
@HardwareGeek said in Dolt:
Mine is how much storage does it use? If permanently preserves every modification to a large, rapidly-changing database, that could chew through storage like mad.
Not as fast as Chia…
-
@HardwareGeek said in Dolt:
@sloosecannon said in Dolt:
My biggest question here would be "is this even remotely performant?"
Mine is how much storage does it use? If permanently preserves every modification to a large, rapidly-changing database, that could chew through storage like mad.
Depends. How often are the changes inserts vs updates or deletes? I'm fairly certain that inserts far outweigh updates for my application.
-
Not too hot at maths either. 19.5 is neither the geometric (11.7) nor even the arithmetic (16.3) mean of those multiples
'd above but somehow I'm pretty sure most people missed the signal lost in the noise:
-
@remi Need to get the mode(s) in there, too.
-
@loopback0 said in Dolt:
@HardwareGeek said in Dolt:
@sloosecannon said in Dolt:
My biggest question here would be "is this even remotely performant?"
Mine is how much storage does it use? If permanently preserves every modification to a large, rapidly-changing database, that could chew through storage like mad.
It'll absolutely do that if the records change often.
If being able to retrieve the state of a record at any point in time is a requirement then however that's implemented will do the same.
At scale storage is (relatively) cheap though.The systems I've seen that had such requirements simply use monthly database backups and a query log.
Apply the most recent backup to whenever, and apply the part of the query log up to the moment, and hey presto, database the way it looked at that specific point in time.
Didn't use insane amounts of data.
Not that it was particularly lightweight either, but the query log was highly compressible at least.Another way is using DRs that can be replayed. I've seen that in other systems to get back up to date after a restore from backup.
-
@Carnage If you want to restore an entire database from a backup, sure.
If you want to instantly see a previous version (or versions) of a record then applying a backup somewhere and replaying queries isn't exactly meeting that requirement.
-
@loopback0 said in Dolt:
@Carnage If you want to restore an entire database from a backup, sure.
If you want to instantly see a previous version (or versions) of a record then applying a backup somewhere and replaying queries isn't exactly meeting that requirement.
YEah, well... I've never really come across a time where that need existed, but cant various big data systems do that as well?
-
@boomzilla I just don't know how to grasp that. I wonder if they intend to refer to themselves or their users.
inb4 why not both
-
@loopback0 said in Dolt:
@Carnage If you want to restore an entire database from a backup, sure.
If you want to instantly see a previous version (or versions) of a record then applying a backup somewhere and replaying queries isn't exactly meeting that requirement.
YEah, well... I've never really come across a time where that need existed, but cant various big data systems do that as well?
I got a ticket today that went something like this...
:luser-1234:The system defrobnicated and encabulated the foobar all on its own without any user intervention! I think this started happening when we changed the default to defrobnicating and encabulating foobars due to COVID.
I looked at the audit logs and :luser-5678:made those changes on February 18th. Also, when the foobar was created in the encabulated state. That hasn't changed at all. Also, we never changed the default frobnication and encabulation settings for foobars due to COVID or anything else.
-
I'm not sure I really see the WTF here. Being able to see a change log for your database can be useful in a lot of scenarios, particularly on internal test systems. If you can do that but the database still looks like a 'normal' MySQL RDBMS to client code then that's great. And most applications aren't database performance limited so even being 10-20 times slower is probably fine, particularly if you're not using it on production systems.
Their implementation might be WTFy. But RabbitMQ (which is close to industry standard status for message brokers) is written in Erlang - if it works, nobody cares what's inside the box.
-
@bobjanova said in Dolt:
But RabbitMQ (which is close to industry standard status for message brokers) is written in Erlang - if it works, nobody cares what's inside the box.
Your argument disagrees with itself. Erlang implementation is a selling point for middleware, not neutral, and far from a detractor.
