Process Markdown files

[dangeRuss]

So we are migrating from Notion to confluence, and while there's no straightforward migration path, looks like someone wrote a script to migrate markdown to confluence.

GitHub - justmiles/go-markdown2confluence: Push markdown files to Confluence Cloud

Push markdown files to Confluence Cloud. Contribute to justmiles/go-markdown2confluence development by creating an account on GitHub.

Notion allows you to export your pages as Markdown.

So far so good. Except there seems to be some bug with special characters in image folders/names in the script. I don't know Go and can't seem to figure it out, but I figure I can preprocess the markdown in python or something.

The problem is I can't seem to find a library that would let you read a markdown file, update the image paths/names and write it back in markdown.

Does anyone know of one?

[Tsaukpaetra]

@dangeRuss said in Process Markdown files:

The problem is I can't seem to find a library that would let you read a markdown file, update the image paths/names and write it back in markdown.
Does anyone know of one?

sed ?

Markdown is, after all, a plain-text format...

[loopback0]

Yeah, I'd probably try sed

[Gribnit]

@loopback0 said in Process Markdown files:

Yeah, I'd probably try sed

If not sed, then awk. And if not awk, give up because I'm not even gonna try to explain lex

[dangeRuss]

@Gribnit said in Process Markdown files:

@loopback0 said in Process Markdown files:

Yeah, I'd probably try sed

If not sed, then awk. And if not awk, give up because I'm not even gonna try to explain lex

So basically you guys don't know any libraries that support this and I should just use regex.

Problem is, I also need to rename the files as well, but might be able to do that with backreferences.

[loopback0]

@dangeRuss said in Process Markdown files:

@Gribnit said in Process Markdown files:

@loopback0 said in Process Markdown files:

Yeah, I'd probably try sed

If not sed, then awk. And if not awk, give up because I'm not even gonna try to explain lex

So basically you guys don't know any libraries that support this and I should just use regex.

I assumed the replacement was sufficiently simple that a library was overkill e.g. removing special characters or replacing all instances of 'patha' with 'pathb'.

@dangeRuss said in Process Markdown files:

Problem is, I also need to rename the files as well, but might be able to do that with backreferences.

How complicated are we talking here? Can you give examples?

[HardwareGeek]

@dangeRuss If sed and awk are potentially good tools for the job, I'd just use the scripting language that combines the best (and worst) features of both, because I can't remember the syntax of either of them. But I'm weird; I actually like perl.

[PleegWat]

@dangeRuss said in Process Markdown files:

Problem is, I also need to rename the files as well, but might be able to do that with backreferences.

awk cannot rename files itself, though it can shell out to mv. Pure shell might be an option as well (bash has its own regex in [[ foo =~ f.* ]]) but that way madness lies..

[dangeRuss]

@HardwareGeek said in Process Markdown files:

@dangeRuss If sed and awk are potentially good tools for the job, I'd just use the scripting language that combines the best (and worst) features of both, because I can't remember the syntax of either of them. But I'm weird; I actually like perl.

Yea perl was in my past too, but these days if I'm doing regex, I will probably use python.

[dangeRuss]

@loopback0 said in Process Markdown files:

How complicated are we talking here? Can you give examples?

Not too crazy, but I haven't seen all the files yet.

So far it's things like [Some Weird Folder/Image 01.jpeg

[loopback0]

@dangeRuss said in Process Markdown files:

@loopback0 said in Process Markdown files:

How complicated are we talking here? Can you give examples?

Not too crazy, but I haven't seen all the files yet.

So far it's things like [Some Weird Folder/Image 01.jpeg

To what?

[dangeRuss]

@loopback0 said in Process Markdown files:

@dangeRuss said in Process Markdown files:

@loopback0 said in Process Markdown files:

How complicated are we talking here? Can you give examples?

Not too crazy, but I haven't seen all the files yet.

So far it's things like [Some Weird Folder/Image 01.jpeg

To what?

To something without special characters. So maybe

Some_Weird_Folder/Image_01.jpeg

[loopback0]

@dangeRuss which is perfect for sed and/or awk. Doesn't even need to be complex as you could just pipe the output back into sed or awk again if needed.
If it's a one-off, it doesn't need to be pretty.

As mentioned already renaming the actual file is just the output of those into mv.

[dangeRuss]

@loopback0 said in Process Markdown files:

@dangeRuss which is perfect for sed and/or awk. Doesn't even need to be complex as you could just pipe the output back into sed or awk again if needed.
If it's a one-off, it doesn't need to be pretty.

As mentioned already renaming the actual file is just the output of those into mv.

As good as I am with bash, sed and awk are still blackboxes for me. Do you have some sample code of how this would look?

[loopback0]

@dangeRuss said in Process Markdown files:

@loopback0 said in Process Markdown files:

@dangeRuss which is perfect for sed and/or awk. Doesn't even need to be complex as you could just pipe the output back into sed or awk again if needed.
If it's a one-off, it doesn't need to be pretty.

As mentioned already renaming the actual file is just the output of those into mv.

As good as I am with bash, sed and awk are still blackboxes for me. Do you have some sample code of how this would look?

It's nearly 11pm here so not if you're in a rush. Google (or someone else here) can help if you are.

[dangeRuss]

@loopback0 said in Process Markdown files:

@dangeRuss said in Process Markdown files:

@loopback0 said in Process Markdown files:

@dangeRuss which is perfect for sed and/or awk. Doesn't even need to be complex as you could just pipe the output back into sed or awk again if needed.
If it's a one-off, it doesn't need to be pretty.

As mentioned already renaming the actual file is just the output of those into mv.

As good as I am with bash, sed and awk are still blackboxes for me. Do you have some sample code of how this would look?

It's nearly 11pm here so not if you're in a rush. Google (or someone else here) can help if you are.

I think I will just do it in python, but still interested in seeing awk/sed solution if/when you have time.