Discourse to NNTP gateway



  • @OffByOne said:

    I'm thinking whether to wrap the badge name in [] to make it stand out that it's not really part of the name, but I'm not sure whether it would be an improvement.

    You may want to add a left-to-right override (U+202D) between the title and the badge name so there is no LTR effects between them (to better match what Discourse is doing).



  • @Onyx said:

    I am now seriously considering using this, if nothing else on mobile. Is there a decent reader on Android?

    A quick DDG turns up http://www.appbrain.com/app/com.almarsoft.GroundhogReader, which looks quite nice.

    Also:
    @OffByOne said:

    When I'm satisfied with my NNTP gateway, I wouldn't mind implementing an IMAP/SMTP frontend too That would make Dicsource on mobile at least fun to use.

    So if you have some patience... ;)
     

    @Onyx said:

    Well, at least until I get my "secret project" rolling... Muahahahaha!

    Ooh, can you tell me? I promise I won't tell anyone else :D



  • @VinDuv said:

    You may want to add a left-to-right override (U+202D) between the title and the badge name so there is no LTR effects between them (to better match what Discourse is doing).

    Thanks. I wasn't sure how to reliably fix that. I'll add that and do proper MIME header encoding while I'm at it.


  • BINNED

    @OffByOne said:

    Ooh, can you tell me? I promise I won't tell anyone else

    I'm intending to port Discoursistency API to Qt, meaning I can then work on local apps for... well, pretty much anything, though I'm limited to Android myself due to dev environment I have available. Other platforms would not be a pain in any case, it's just UI work once the API is done, the C++ bit just needs a recompile and you're good to go.

    I'll probably have to badger people on the UI design front though since I'm not really a good GUI designer.

    Hopefully I'll be able to get the time to finish the backend until the end of the month, waiting for Qt 5.4 and native Android control set coming with that release (Dec 2 on official roadmap). Also, WinRT support.



  • @VinDuv said:

    You may want to add a left-to-right override (U+202D) between the title and the badge name so there is no LTR effects between them (to better match what Discourse is doing).

    Fixed, thanks for pointing it out.

    Names are handled completely correctly now: if a part of the name falls outside ASCII, I append an LRO to be on the safe side. < and > are substituted with 〈 and 〉. < is changed because it cannot be escaped in an email address and > is changed for symmetry.
    *looking hard at your displayed badge* ;)


  • FoxDev

    having issue with sockbot right now... PMs send it into an infinite loop getting notifications...

    whyyyyyyyyyyy?



  • Judging by your "From" column, it seems that my long-name non-exploit is somewhat successful... What do the semicolons do in the protocol?

    @Onyx said:

    I'm intending to port Discoursistency API to Qt

    Good luck, it being about a few percent done. I'm really trying to at least wrap up the raw request part, but time issues.


  • BINNED

    @Maciejasjmj said:

    Good luck, it being about a few percent done. I'm really trying to at least wrap up the raw request part, but time issues.

    Heh. Time issues. You're at a few percent, I'm at 0 still :P



  • @Maciejasjmj said:

    Judging by your "From" column, it seems that my long-name non-exploit is somewhat successful... What do the semicolons do in the protocol?

    List multiple addresses. Think of a To: field in an email that looks like To: Cool Polish dude <maciejasjmj@what.thedailywtf.com>; B?????n guy <offbyone@what.thedailywtf.com>.

    At least your exploit didn't cause my gateway to drop any tables ;)

    I've tinkered a bit more with "From" name generation and escaping special characters in names. Your "From" entry now looks like this (screenshot from Thunderbird):

    In the top pane, Thunderbird truncates your username at the ; and surrounds it with quotes. In the "From" line in the preview window, it shows your username as intended.

    I guess the Discoursistency in the way your username gets rendered is client specific.

    Yup, confirmed with slrn:

    Btw, the raw From-line for that post is From: Maciejasjmj | =?UTF-8?B?JyBPUiAxPTE7IERST1AgVEFCTEUgVERXVEY7IC0t?= =?UTF-8?B?IDxrYmQ+WFNTIEF3YXJkPC9rYmQ+?= <Maciejasjmj@what.thedailywtf.com>.



  • My title was also getting cut off: riking &lt



  • @riking said:

    My title was also getting cut off: riking &lt

    Yes, your title is the Worst of the Worst™ with respect to converting it to a valid email name.
    I encode your name, so there is no bare ; in the address the Usenet client receives. It seems most clients first decode and then split on certain characters (; being one of them). I have found no way yet to get a ; as a regular character in a name, all clients I tested with treat it as special in some way.

    There are some more things I can try, like prepending a \ or surrounding with quotes. As a last resort I can substitute it with a similar unicode character that has no special semantic meaning.
    Since it's a cosmetic bug (and one I'm not sure I can work around on the server side), I'm going to focus on more pressing issues first, like being able to post and providing an interface to the profile, notifications, ...

    Right now I'm refactoring the code and I'll put it on Github ASAP, so I have a decent way to track bugs and feature requests.


  • Garbage Person

    I think you've developed the definitive Discourse UI.

    As far as handling PM's, you should clearly sidechannel them to SMTP/POP3 (or IMAP) just like real NNTP.


  • Discourse touched me in a no-no place

    @OffByOne said:

    Since it's a cosmetic bug

    Well it does interface with Discourse. Speaking of which....

    @OffByOne said:

    Github ASAP, so I have a decent way to track bugs and feature requests.

    Doing It Wrong(tm) - you should have an instance of Discourse to track bugs. It's really good for that. Then you'd be able to connect to your bug tracker using NNTP in true WTF style.



  • @OffByOne said:

    As a last resort I can substitute it with a similar unicode character that has no special semantic meaning.

    Here, take this: ;


  • FoxDev



  • @Weng said:

    I think you've developed the definitive Discourse UI.

    *blush* Thanks 😃

    I hope it's useful to other people besides me. My sofa laptop laptop is quite underpowered, Discourse via the web-interface is just painful. Thunderbird (or slrn if I feel nostalgic) is much friendlier for that computer.

    @Weng said:

    As far as handling PM's, you should clearly sidechannel them to SMTP/POP3 (or IMAP) just like real NNTP.

    PM's in Discourse are just topics with some privacy sprinkled on top of them, if I understand it correctly. They map quite nicely to the existing newsgroups representation.

    I'm going to add some kind of "meta" group that allows you to control your profile, bookmarks, ... too. I'd prefer to keep everything inside 1 protocol. It puts less cognitive load on the user and is easier to implement (no separate parts that have to coördinate their actions).
    I'm pleasantly surprised actually how naturally Discourse features map to the NNTP protocol.

    Speaking of SMTP+IMAP, I want to try that when I've got a reasonably complete implementation of NNTP.
    I've taken care to structure my code in a modular way, so adding those protocols should just mean slapping an IMAP and SMTP server in front of the middle- and backend code I already have.

    After that, maybe I'll implement that SNMP frontend @PJH requested ;)



  • @loopback0 said:

    @OffByOne said:
    Since it's a cosmetic bug

    Well it does interface with Discourse. Speaking of which....

    You've got a point there, but it's a poor craftsman who blames his tools for a shitty result...
    I try to make something nice to use, even though the backend I have to work with is... erm... Discobuggy.

    @loopback0 said:

    @OffByOne said:
    Github ASAP, so I have a decent way to track bugs and feature requests.

    Doing It Wrong(tm) - you should have an instance of Discourse to track bugs. It's really good for that.

    So I've heard. I'll start using Discourse as a bug tracker around the same time that I start using Redmine for casual conversations ;)

    @loopback0 said:

    Then you'd be able to connect to your bug tracker using NNTP in true WTF style.

    That sounds strangely appealing. I could also whip out a Github backend for my gateway and achieve the same thing.
    When my IMAP+SMTP frontend is implemented, I'd even be able to interface to my bugtracker using my K-9 on my phone 😃



  • @OffByOne said:

    I'm pleasantly surprised actually how naturally Discourse features map to the NNTP protocol.

    You should post that over at meta.d. Somebody just might have a heart attack.



  • Does your NNTP client support killfiles?



  • who knows


  • BINNED

    @OffByOne said:

    I'm pleasantly surprised actually how naturally Discourse features map to the NNTP protocol.

    So we found out that, despite all the Jeff's hate of traditional forums and, even worse, tree-like structure of NNTP and sites like Reddit, Discourse actually maps nicely to a tree structure because that's the way online discussions work!

    Oh, wait, that's what we've been saying all along. It's just that JDGI.


  • Discourse touched me in a no-no place

    @OffByOne said:

    After that, maybe I'll implement that SNMP frontend @PJH requested



  • @HardwareGeek said:

    Does your NNTP client support killfiles?

    My clients (Thunderbird, Pan, slrn) all do. My gateway doesn't, because it's a server. I could implement some filtering/killing mechanism.
    I consider that low priority though, since most (all?) NNTP clients already have that functionality onboard.



  • @PJH said:

    http://pen.iana.org/pen/PenApplication.page

    @From that page said:

    Usually, a single PEN is granted per organization

    Doesn't that mean that tCotCDCK needs to fill out the application? Maybe we should log a feature request in their bugtrac... bwahaha, couldn't keep a straigt face! 😆

    I'd be happy to implement my frontend so it conforms to their MIB, but I'm not looking forward to writing a MIB myself.


  • Discourse touched me in a no-no place

    @OffByOne said:

    Doesn't that mean that tCotCDCK needs to fill out the application?

    Not necessarily. Your program, your enterprise number.

    @OffByOne said:

    but I'm not looking forward to writing a MIB myself.

    They're really not that bad to write from scratch... 👿



  • Also, WTF? 3 posts show up blank, replying and clicking the quote button gives the Eternal Spinner of Death.

    The raw shows the posts contents though.

    @discoursebot, what do you say?



  • @OffByOne - Days Since Last Discourse Bug: 0



  • @PJH said:

    They're really not that bad to write from scratch... 👿

    Not if you get someone else to do it for you 😈



  • @HardwareGeek said:

    You should post that over at meta.d. Somebody just might have a heart attack.

    Heh. They designed Discourse that way. I'm merely pointing out an observation ;)

    I don't have an account on meta.d (yet?). Maybe when DiscoNews is mature enough, I'll create a login there so I can tell them about how Discourse is a wonderful datastore for an NNTP server.

    All usenet clients I know of use infiniscroll in the newsgroups and article views, not pagination. They can't be mad for a frontend that Does It Right™, right? 😃



  • @OffByOne said:

    My gateway doesn't, because it's a server.

    My bad. I'm not sure why I thought it was the client. Now that you mention it, I remember searching CPAN to find a NNTP server module.

    @OffByOne said:

    My clients (Thunderbird, Pan, slrn) all do.

    I didn't know Thunderbird did. I have used it for NNTP, but rarely. It caches stuff locally that I don't necessarily want cached locally, and it takes a bit of poking about in odd places (or used to under XP) to find and purge it.


  • Discourse touched me in a no-no place



  • @PJH said:

    >OffByOne:
    DiscoNews

    Definition not found.

     

    @OffByOne said:

    I can mock a DiscoNews sock_module in Perl and use that to interface with your core files, so that's not really a problem



  • @PJH said:

    Definition not found.

    I'll add it to the Discopædia once I've put it on Github.



  • I am intrigued by this thread because while I've known for a long time that discussions mapping to threading works surprisingly well, this is the first time I've seen a UI for it that actually doesn't feel like it sucks.

    vB and IPB used to have threaded replies, both dropped them. I've thought about doing it before myself but the UI always sucked, partly because users have a habit of clicking reply to the wrong thing - but that's a problem here anyway ;)

    Seriously though... I would kill for a forum software that solved the UI problem well on this one.



  • @Arantor said:

    I am intrigued by this thread because while I've known for a long time that discussions mapping to threading works surprisingly well, this is the first time I've seen a UI for it that actually doesn't feel like it sucks.

    Usenet has been around since 1980, threading viewers since... I don't know, a quick Google didn't turn up anything useful 😛

    @Arantor said:

    though... I would kill for a forum software that solved the UI problem well on this one.

    Not trying to be pendantic here, but do you mean "software" as in "web-based frontend" or as in "application used to interact with the forum"?
    My gateway uses Discourse as a data store and leaves the UI to the NNTP client you want to use. It's not a UI in itself.

    I'm doing my best to make the thing modular, both on the backend and on the frontend. Whatever JSON gets returned by Discourse is transformed into a data structure that should work reasonably for any message-based backend, not specifically Discourse. That data structure then gets passed to the frontend, which doesn't have to know anything about Discourse specific data structures.

    This means that it shouldn't be exceedingly hard to write a backend plugin that would interface with for example vB and have the frontend(s) deal with it without extra effort.
    As long as the backend provides a means to retrieve messages and their hierarchical meta-information (to which category/topic do they belong, what is the parent message) it shouldn't be very difficult to make DiscoNews talk to it.

    On the frontend side, I'm first going to implement an IMAP/SMTP module. After that, I'm going to see if I can write a simple HTTP frontend with pagination and no Javascript requirements :D (If I'm going to piss off Jeff, I might as well go all out).

    DiscoNews development has stalled because work and Real Life. I hope to resume working on it this weekend.



  • Specifically: I want a piece of software I can run on a website that does the threading thing properly. I don't want discrete clients for this.

    There are people I know that would quite happily use such a forum software if it did it sanely, but none of the implementations ever feels sane to me.

    As far as your modularisation goes; most of the flat forum software (which includes vB, IPB, XF, SMF, phpBB, and going forward MyBB) doesn't store the hierarchical stuff. They are actively phasing it out (vB, IPB, MyBB have supported it, all have phased it out because 'that's not how the web works' which is utter bullshit)

    And yes, I know about Usenet... I just never tried it for myself 😱


  • FoxDev

    @Arantor said:

    And yes, I know about Usenet... I just never tried it for myself

    .... it is mostly overrated, although i really should sort out that billing dispute with my usenet gateway...

    They contend i owe them six months service charges plus late fees and interest, i contend i canceled my service with them before that billing period began.



  • You mean like most internet forums? 😆

    But seriously, I reckon a decent forum software that handled nesting stuff and whatnot would be a serious player in the forum market - potentially.


  • I survived the hour long Uno hand

    Arantor for WTForum project lead!

    🏁



  • Not at this time. It will be a thing - but not right now. I need to fix a lot of things in my head before I dare risk that. Right now I am way too insecure in my self-trust to make that a thing.


  • ♿ (Parody)

    @Arantor said:

    But seriously, I reckon a decent forum software that handled nesting stuff and whatnot would be a serious player in the forum market - potentially.

    The real challenge is simply figuring out how to make the UI not suck. And that goes beyond the web. I mean, there is inherent suckiness in threaded and non-threaded views. As a wise boss once said, All babies are ugly, but you have to pick one.



  • Yup, and that's been my contention with it. But newsreaders seem to manage this, yes?



  • Feature Request: Threaded view that looks like the Tech Web from Civilization: Beyond Earth. Or a glorified Visio flow chart, but Civ is better.


  • ♿ (Parody)

    @Arantor said:

    But newsreaders seem to manage this, yes?

    Seems like an eye of the beholder problem. The biggest problem with threaded interfaces for me is that it makes figuring out and getting to what's unread a PITA. I like the way Discourse tries to handle this by allowing me to click on stuff to see what was replied to (and what's replied to the item I'm looking at).



  • Interesting perspective since that's not what I perceived the biggest UI problem with threading to be - mine was about getting users to actually click on the thing that indicated the proper hierarchy for their post.

    I mean, consider this: user replies to post A, quotes post B, is it not implicitly a descendent of both? If post B is a reply to post A, fine, but when they're not... what then?


  • ♿ (Parody)

    @Arantor said:

    mine was about getting users to actually click on the thing that indicated the proper hierarchy for their post.

    @Arantor said:

    If post B is a reply to post A, fine, but when they're not... what then?

    You mean...a complicated hierarchy with multiple parents? Yeah, I don't know how you solve that. Show it in multiple places? Is this really that much of a problem?

    I mean, from the perspective of creating a 100% accurate graph of a conversation, I can understand what you seem to be getting at. But from a user's perspective, the importance of threading is keeping context. And the downside I guess is that sometimes people change the subject?


  • Discourse touched me in a no-no place

    @Arantor said:

    I mean, consider this: user replies to post A, quotes post B, is it not implicitly a descendent of both? If post B is a reply to post A, fine, but when they're not... what then?

    Usenet used the In-Reply-To header, where it listed a set of message IDs of parent posts. I always assumed it showed parent, grandparent, and so on, in order.

    the Discourse preferred model of gathering multiple replies into one post doesn't fit into that mold, however. In Usenet, you'd just see one parent in a threaded newsreader, but people weren't assumed to be stupid, and post IDs weren't conserved a scare resource, so people were able to handle multiple separate replies. Perhaps Jeff wasn't smart enough to handle that.



  • That's kind of my point, yes... for threading to be meaningful, context needs to be kept and the whole 'multi-parent' thing is kind of problematic in general but it's almost a symptom of 'getting users' to click on the right thing to express what they mean. They're all interlinked.

    Discourse for all its faults, is generally quite sane about this, though not perfect because Discourse and users.



  • So that's specifically going off 'I reply to this even if I quote that', I can see that. That's logical, yes.

    And yes, the whole 'multiquoting' really doesn't fit that model which is why I'm bringing it up ;) Especially as it can easily cause what amounts to reparenting, which is what makes it seriously interesting to me to contemplate.


  • Discourse touched me in a no-no place

    I don't know what to do. Even in Discourse, you can only see one "this is a reply to" top-of-post indicator, so I would probably use the same model NNTP uses, and say a post can only be a reply to one other post for treeview purposes.


Log in to reply