C-style #include in TypeScript... possible?

blakeyrat

I have a TypeScript project which right now consists of about 6 modules and 1 type definition file. For reasons of compression and obfuscation, I'd like the final built version to be just one single module... right now the TypeScript build system adds a lot of boilerplate to support the modules, and it also "leaks" module names as they do not get properly obfuscated.

What I'd like to do is have a TypeScript project which consists of several files, for my own code organization purposes, but when built they are all put into a single module. Naturally, I'd like this combining to happen before the code is run through Uglify.js so it'll correctly detect and obfuscate local variables.

The alternative is to get some kind of C-style #include I could use with the TypeScript compiler, so that it combines all the files before doing its build process. That would achieve the same goal I think.

Currently the project is built with Gulp, so any tools used should work with the Gulp ecosystem.

Has anybody done anything like this before?

blakeyrat

This concept seems interesting:

Search | Packt Subscription

Search over 7,500 Programming & Development eBooks and videos to advance your IT skills, including Web Development, Application Development and Networking

I wonder if I could "hack" this behavior to do what I want?

DCoder

@blakeyrat what does your TS code look like now? Is it properly structured into modules that explicitly export their public APIs and import their dependencies, or is it coded in the traditional JS way "all the external stuff I need is already on the page, loaded via other <script> tags, when I am executed"?

If your code uses proper modules (which have other architectural benefits besides bundling), you can tell the TS compiler itself to bundle everything into a single .js file that you can later run through Uglify.js. But modularization will leave the original module names and exported APIs, you will need to look for some external obfuscation solution to mangle those if this is really important.

The link you posted to module merging/declaration merging will not help here. That link is trying to explain that TypeScript will play nice with multiple declarations talking about the same thing. For example, if you have two .d.ts files from two different jQuery plugins, one saying that "jQuery has a method called .draggable()" and the other saying "jQuery has a method called .datetimepicker()", TypeScript will use information from both those declarations when figuring out what methods are available on a jQuery object. It will not say "hey, this jQuery type has already been defined earlier!". This has no effect on actual TS code you write, only on declarations of external stuff.

masonwheeler

@blakeyrat said in C-style #include in TypeScript... possible?:

For reasons of ... obfuscation

I'mma stop you right there. Please don't. It's a big hassle for you, to create a big hassle for your users, that provides very little actual benefit for anyone involved.

anotherusername

Minified code should be sufficiently obfuscated, and you should end up with a source map file so that you can use the unminified source when debugging.

Rhywden

@masonwheeler Not to mention that this will either increase the size of files to be loaded - or the "obfuscation" will be stripped right out again while the build process minimizes everything.

blakeyrat

@dcoder said in C-style #include in TypeScript... possible?:

what does your TS code look like now?

I have a .d.ts file to define by types, and a bunch of classes that are files with a few imports at the top and then:

export class FooBar { }

The only file I have that isn't an exported class looks like this:

import { FooBar } from "./foobar";

var fooBar = new FooBar();

For the purpose of initialization on the page.

So I guess I'm using "exported classes" instead of modules? I'm unclear about how all this fits together, because I'm not using anything that resembles the sample code in that book exerpt.

@dcoder said in C-style #include in TypeScript... possible?:

Is it properly structured into modules that explicitly export their public APIs and import their dependencies,

I am using import/exports, but I guess at the "class level" and not at the "module level"?

@dcoder said in C-style #include in TypeScript... possible?:

If your code uses proper modules (which have other architectural benefits besides bundling), you can tell the TS compiler itself to bundle everything into a single .js file that you can later run through Uglify.js.

It does that now, naturally, but the module names remain un-obfuscated because Uglify.js is too stupid to figure out that they are actually internal to my product and not something anything else on the site will ever touch.

I understand the architectural benefits of modules, of course. But I don't want the final built product to contain any remnants of them.

@dcoder said in C-style #include in TypeScript... possible?:

But modularization will leave the original module names and exported APIs, you will need to look for some external obfuscation solution to mangle those if this is really important.

Right; do you know of a tool that does that and works with Gulp? That's pretty much exactly the question I'm asking.

@dcoder said in C-style #include in TypeScript... possible?:

he link you posted to module merging/declaration merging will not help here.

Yeah I see that now.

blakeyrat

@masonwheeler said in C-style #include in TypeScript... possible?:

I'mma stop you right there.

Please don't be a dick in help threads.

This product is equivalent to Google Analytics, Adobe Analytics, etc. It's perfectly natural, normal and expected to deliver this class of product in a single obfuscated file.

blakeyrat

@anotherusername said in C-style #include in TypeScript... possible?:

Minified code should be sufficiently obfuscated, and you should end up with a source map file so that you can use the unminified source when debugging.

Sigh. Pedants.

For the purposes of this question "obfuscated == minified", ok? Christ.

And yes I do currently get a source map generated for me.

@rhywden said in C-style #include in TypeScript... possible?:

Not to mention that this will either increase the size of files to be loaded - or the "obfuscation" will be stripped right out again while the build process minimizes everything.

Minimization is obfuscation. It's the same thing. If I say I'm driving a car, you don't have to come in here and say "well you should consider driving a pickup truck", everybody understands that they are the same class of thing and the word "car" includes both.

Unless someone specifically says "I am obfuscating a script without minifying it" you should just assume that they're obfuscating it by minifying it, because that's the only rational interpretation. Yes I understand it's technically possible to do one without also doing the other, but no rational person would do that ever.

Ok? Sheesh.

Rhywden

@blakeyrat Obfuscation can also include additional function calls, redirects, extra variables or even self-generating code.

Thus minimization is only a subset of obfuscation.

dkf

@rhywden said in C-style #include in TypeScript... possible?:

Thus minimization is only a subset of obfuscation.

Some of the more promise-heavy code samples I've seen here have struck me as being rather obfuscated (in the original programmer's brain) despite clearly not being minimised in the slightest…

Rhywden

@dkf said in C-style #include in TypeScript... possible?:

@rhywden said in C-style #include in TypeScript... possible?:

Thus minimization is only a subset of obfuscation.

Some of the more promise-heavy code samples I've seen here have struck me as being rather obfuscated (in the original programmer's brain) despite clearly not being minimised in the slightest…

You could also include the code for a compression algorithm (I'm sure there are some small-ish ones available), use that one to compress part of your code. Divide the resulting string into random-sized parts, sprinkle those liberally over your files (but in a way that they are available at page start). Then re-assemble and decompress everything at the start and eval() the heck out of it. Bonus points for watching for the invocation of dev-tools which inject random gibberish into said string parts.

blakeyrat

Guys if you don't have an answer to the question or any information useful to me, please post elsewhere.

Rhywden

@blakeyrat Just told you how to obfuscate your code in a really nice way.

DCoder

@blakeyrat said in C-style #include in TypeScript... possible?:

Right; do you know of a tool that does that and works with Gulp? That's pretty much exactly the question I'm asking.

Sorry, I haven't experimented with such aggressive minification in a really long time. While your idea makes sense, I think people exporting public APIs don't want them to get mangled, and that conflicts with what you are trying to achieve. Uglify seems to have a harmony branch for minifying ES6 code – that might help, but I haven't tried it.

blakeyrat

@dcoder Well like I said I'd also be ok going the low-tech route and using #include (or something equivalent) to put the entire project in one big module in the first place. Since that's really what it is: one big module.

DCoder

@blakeyrat This approach might work then, TS calls it "internal modules" or namespaces. Instead of ES6 modules, explicitly put all your classes in the same namespace:

// file mod1.ts
namespace me {
  export var foo = 'test from mod1';
}

// file mod2.ts
namespace me {
  export function bar(input: string) { return "called bar(" + input + ")"; }
}

// file main.ts
/// <reference path='mod1.ts'/>
/// <reference path='mod2.ts'/>

alert(me.bar(me.foo));

Build with tsc main.ts --outfile script.js to get:

var me;
(function (me) {
    me.foo = 'test from mod1';
})(me || (me = {}));
var me;
(function (me) {
    function bar(input) { return "called bar(" + input + ")"; }
    me.bar = bar;
})(me || (me = {}));
/// <reference path='mod1.ts'/>
/// <reference path='mod2.ts'/>
alert(me.bar(me.foo));

Then Uglify will happily mangle it all, leaving only the outermost function(){}() as past module boundaries:

var n;!function(n){n.t="test from mod1"}(n||(n={})),function(n){n.o=function(n){return"called bar("+n+")"}}(n||(n={})),alert(n.o(n.t));

blakeyrat

@dcoder Awesome tip, I'll give it a try.

blakeyrat

@dcoder Ok if I wrap a class in a Namespace, and then use an import from another file I get this error:

File XXX/library.ts is not a module

But it is a module, it's just a module inside a namespace?

If I move the import inside the namespace, I get:

Import declarations in a namespace cannot reference a module

So I'm a bit stuck.

EDIT: I also noticed that for the exported classes inside a namespace, I get this error:

TypeScript error: source/init.ts(1,10): Error TS2305: Module '"source/tracker"' has no exported member 'Tracker'.

Which implies that exports inside a namespace aren't actually exporting stuff? Not sure what's going on.

DCoder

@blakeyrat Your previous code with import {foo} from 'bar' was using external modules. My suggestion with module {} and /// <reference path=...> uses namespaces. The two mechanisms are not really compatible with each other. As I see it, you need to remove all import statements and add /// <reference path statements instead.

blakeyrat

@dcoder Yeah thanks, sorry I didn't update this thread but I figured that out eventually on my own. Once everything's in the same namespace, there's obviously no need to import it, that's kind of a "duh" when you think about it.

Now my only problem is in my init.ts the initializing code looks like this:

import { FooBar } from "./foobar";

var tracker = new FooBar();

Where "FooBar" is the "global class" (so to speak) of the product.

TypeScript is giving me the error "file foobar.ts is not a module" and I'm not sure why. It is a module, it's just inside a namespace...

DCoder

@blakeyrat What happens if you change init.ts like this?

/// <reference path="foobar.ts" />

var tracker = new YourNamespaceName.FooBar();

blakeyrat

@dcoder Well it builds, give me a few minutes and I'll tell you if it runs.

Thanks so much for your help.

EDIT: it builds but when I run it the browser says:

Uncaught ReferenceError, YourNamespaceName is not defined.

Do TypeScript namespaces turn into a JS object, or are they erased during type erasure?

EDIT: oh no wonder, if somehow "optimizes-out" the entire import and none of the namespace is imported. Huh. Weird that TypeScript compiles without pointing that out in any way.

DCoder

@blakeyrat You said you have a .d.ts file in this project, can you try moving it outside the project and building without it? I am not sure what else could cause such problems, I'm afraid.

blakeyrat

@dcoder said in C-style #include in TypeScript... possible?:

You said you have a .d.ts file in this project, can you try moving it outside the project and building without it?

Not sure how you propose to do that, it contains interfaces that the rest of the code needs to build properly...

I did add a declare keyword to it because TypeScript bitched at me before it was there:

declare namespace YourNamespaceName {

It could also be a bug/problem with the Gulp "Tsify" plugin I'm using to run the TypeScript compiler.

EDIT: the "official" (I guess?) documentation implies you just need to add the reference files and it all just magically works. https://www.typescriptlang.org/docs/handbook/namespaces.html

My experience is otherwise. The reference path declaration in my build environment is doing, AFAICT, nothing at all. Even weirder, they appear unaltered in the output of the TS compiler...

blakeyrat

@blakeyrat Ok I've determined I'm too stupid to make this work. I'll do something else for now and revisit later.

DCoder

@blakeyrat Ah, that's not right. .d.ts files are similar to C library headers – they tell the TS compiler "this stuff will exist at runtime, even if you can't see it now, I promise". The TS compiler trusts what you say and won't generate actual code for that stuff. These files are meant primarily for stuff you get from the browser JS runtime/Node runtime/other scripts running in the same scope, such as window, document.createElement, or jQuery.get(). You shouldn't use .d.ts to describe your own code, that's what plain .ts files are for.

blakeyrat

@dcoder Hm. I recall they were just described as "data type definition files" with no insinuation that it's for data types outside your app. But yeah I can refactor that; there's no reason these HAVE to be in a .d.ts.

blakeyrat

@blakeyrat For the record, I did some refactoring and folded the types that were in the .d.ts file into the class files where they were used. I still can't figure out how to get this compiled correctly. What's even weirder is that when I follow the examples in the official documentation exactly it still doesn't work, which makes me think it's some problem with the Gulp build instructions and not TypeScript itself.

blakeyrat

I think this is a flaw in tsify, based on this bug report:

reference path vs require · Issue #69 · TypeStrong/tsify

Hi, this isn't really a tsify issue as more a (maybe stupid) support question about how to handle typescript modules. By using tsc as transpiler I am able to define a custom order of script inc...

The author of tsify seems to allude to the fact that it does not support namespaces... at least that's how I'm reading his statement. Unfortunately he doesn't say what the alternatives to tsify are in that case...

justhelpingout

What you probably want is to run everything through TS with the output targeting ES2015+ (or with just imports / exports preserved, if you are targeting <IE11 ... I think that's possible). Then pipe that into RollupJS (which knows how to optimize your code into a single bundle). Finally pipe that into Uglify.

Alternatively, ClosureCompiler now understands import / export too, so you could bundle and minify in one go and just have a TS -> CC pipeline.

DCoder

@blakeyrat I hate the JavaScript ecosystem for shit like this. Toolchains proliferate like rabbits, workflows are engineered by Rube Goldberg/M. C. Escher, and best practices change more often than socks.

Back when I was dealing with Gulp, I used gulp-typescript to compile TS to JS – by my understanding, if you do that before Browserify is invoked, you won't need tsify. But like @justhelpingout is saying, there are other options and they may work better, I don't want to suggest that my approach is the right way.