Wherefore `from` in LINQ?


  • Winner of the 2016 Presidential Election

    Pretty much what the title says: Why does LINQ syntax use from? My google-fu is weak on the matter.



  • @Dreikin said in Wherefore `from` in LINQ?:

    Pretty much what the title says: Why does LINQ syntax use from? My google-fu is weak on the matter.

    What do you mean, "why" - the rationale behind picking this particular keyword (which is because LINQ tries to mimic SQL), or what it's used for (to, well, point out which collection you're selecting from)?

    Also, LINQ mock-SQL syntax sucks. Extension methods make much more sense.


  • Winner of the 2016 Presidential Election

    @Maciejasjmj said in Wherefore `from` in LINQ?:

    the rationale behind picking this particular keyword (which is because LINQ tries to mimic SQL)

    This one. for would seem to make more sense in terms of making it easy to read because of the different select placements, and because LINQ uses x in y rather than y as x. Although I suppose it is exactly that: because it's the matching SQL keyword. Which then leads to why x in y instead of y as x?


  • Winner of the 2016 Presidential Election

    @Maciejasjmj said in Wherefore `from` in LINQ?:

    Also, LINQ mock-SQL syntax sucks. Extension methods make much more sense.

    I have not yet developed an opinion on that. Most of what I've looked at so far uses the mock-SQL syntax, so I don't have much experience with the extension method syntax.


  • sockdevs

    @Dreikin said in Wherefore `from` in LINQ?:

    Which then leads to why x in y instead of y as x?

    To imitate foreach (var x in y) would be my guess.

    @Dreikin said in Wherefore `from` in LINQ?:

    @Maciejasjmj said in Wherefore `from` in LINQ?:

    Also, LINQ mock-SQL syntax sucks. Extension methods make much more sense.

    I have not yet developed an opinion on that. Most of what I've looked at so far uses the mock-SQL syntax, so I don't have much experience with the extension method syntax.

    Generally, I use comprehension for large and complex queries, and extension methods for simple queries.

    I find the join comprehension syntax easier to use than the extension method


  • Winner of the 2016 Presidential Election

    @RaceProUK said in Wherefore `from` in LINQ?:

    To imitate foreach (var x in y) would be my guess.

    Yeah, that's what I figured. It's like they couldn't decide between for x in y and from y as x and instead compromised by taking one keyword from each. Which baffles me because that doesn't seem to fit with all the other effort put into making it nice to use.


  • sockdevs

    @Dreikin I'd guess they went for from x in y over for x in y not just for the pseudo-SQL aspect, but also to avoid complications with the existing for (;;) construct.

    Remember that when LINQ comprehension syntax was added, it was added with contextual keywords… which means they could have used for x in y after all.

    Hm.



  • The reverse sql like syntax was when Linq-to-sql was a fad around about 2007. I would use the lambda syntax instead.



  • @RaceProUK said in Wherefore `from` in LINQ?:

    I find the join comprehension syntax easier to use than the extension method

    I'unno, it seems simple. Other table, key A, key B, selector. YMMV of course, but it looks scarier than it is.

    Besides, you need to know the method syntax anyway once you start rolling your own expression trees (which is sometimes immensely useful for things like generic repositories - or rather, when you step outside what C#'s generics can do for you).


  • sockdevs

    @Maciejasjmj Never had to roll by own expression trees; never had the need.

    May decide to have a play though, just in case.



  • @RaceProUK said in Wherefore `from` in LINQ?:

    May decide to have a play though, just in case.

    Like all reflection-related stuff, it's an amazing way to shoot yourself in the foot. But sometimes it's the right tool for the job.

    For example: in my codebase, most entities are tied to a particular "environment". There are several types of those environments, and all those types reside in different tables, without normalization (not my call, don't yell). An environment (any environment, really) can be "live" or not - and most of the time we'll only want to query for entities from a "live" environment.

    Now what do you do? You can't expose an IsLive property from the underlying environment, since Entity Framework will not turn that property access into a query. Interfaces don't get you too far either. So attributes to the rescue!

    First, our entity has a [Environment] navigation property:

    public class Widget
    {
        //...
        [Environment]
        [ForeignKey(...)]
        public GreenEnv ManagingEnv { get; set; }
    }
    

    Then, the system has a [Live] boolean property:

    public class GreenEnv
    {
        //...
        [Live]
        [Column(...)]
        public bool IsLive { get; set; }
    }
    

    Finally, as we build our generic repository, we build the argument for Where:

    var envProperty = typeof(TEntity).GetProperties().First(x => x.IsDefined(typeof(EnvironmentAttribute)));
    var liveProperty = envProperty.PropertyType.First(x => x.IsDefined(typeof(LiveAttribute)));
    
    var parameter = Expression.Parameter(typeof(TEntity)); //our "x" in a lambda
    var envPropertyAccess = Expression.Property(parameter, envProperty); //x.ManagingEnv
    var livePropertyAccess = Expression.Property(envPropertyAccess, liveProperty); //x.ManagingEnv.IsLive
    this.LiveQuery = Expression.Lambda<Func<TEntity, bool>>(livePropertyAccess, parameter);  //x => x.ManagingEnv.IsLive
    

    Then all the inner Query or GetOne methods of the repository work from

    public virtual IQueryable<TEntity> LiveQueryable => Queryable.Where(this.LiveQuery);
    

    automatically filtering the entities to the live environments, which gets passed to Entity Framework without any problems.


  • Discourse touched me in a no-no place

    @Maciejasjmj said in Wherefore `from` in LINQ?:

    TEntity

    Did you mean Widget or is there some sort of inference rule going on here?


  • Winner of the 2016 Presidential Election

    @dkf said in Wherefore `from` in LINQ?:

    @Maciejasjmj said in Wherefore `from` in LINQ?:

    TEntity

    Did you mean Widget or is there some sort of inference rule going on here?

    Neither, I think. TEntity is the EF version of foo, usually used as a generic type parameter and indicating an argument should be of some EF entity type.



  • @dkf said in Wherefore `from` in LINQ?:

    @Maciejasjmj said in Wherefore `from` in LINQ?:

    TEntity

    Did you mean Widget or is there some sort of inference rule going on here?

    Yeah, it's a generic repository:

    public class Repository<TEntity>
    

    and TEntity might be Widget or something else. Which is kind of the point - we have a lot of different tables that have those environments, and we want to have a simple solution to filter them all regardless of the actual type.


  • Discourse touched me in a no-no place

    @Maciejasjmj said in Wherefore `from` in LINQ?:

    Yeah, it's a generic repository:

    Type variable from context? OK, gotcha.


  • sockdevs

    @Dreikin said in Wherefore `from` in LINQ?:

    Pretty much what the title says: Why does LINQ syntax use from? My google-fu is weak on the matter.

    well for was already a language keyword, so can't use that or risk accidentally introducing ambiguity.

    as for select.... I think that's because they wanted the fluent style to flow simmilarly to the extension methods

    or it could be the developer that did the demo used from and it stuck.

    demos can be like that.


  • sockdevs

    @accalia said in Wherefore `from` in LINQ?:

    well for was already a language keyword, so can't use that or risk accidentally introducing ambiguity.

    Only if you don't take context into account. It could have been done so that if for appears in the RHS of an expression, it's a LINQ for, else it's a loop for.


  • Discourse touched me in a no-no place

    @RaceProUK said in Wherefore `from` in LINQ?:

    It could have been done so that if for appears in the RHS of an expression, it's a LINQ for, else it's a loop for.

    Now, how are you going to teach that subtle nicety to new users? Take your time…


  • sockdevs

    @dkf I'd expect people to pretty quickly figure out that for (;;) {} and var q = for x in y are very different statements :P


  • Discourse touched me in a no-no place

    @RaceProUK Have you ever done teaching of new users? :scream_cat: The more complicated and contextual things are, the harder it is to punch it in through the thickheaded inobservantness that predominates. In some cases, resorting to physical :punch: feels so tempting (I'm not really all that good at teaching at the whole class level; one-on-one is different :innocent:). Sure, some people get it without trouble. Others… don't.

    Keep things simple. Keep things meaning the same thing in all places.


  • Winner of the 2016 Presidential Election

    @dkf said in Wherefore `from` in LINQ?:

    Keep things simple. Keep things meaning the same thing in all places.

    From that level of understanding, it does mean the same thing in both places.

    Edit:

    Actually, it means the same (in C# terms) as foreach (x in y) at the basic level of understanding. Which I also think would be better than from x in y.


  • Winner of the 2016 Presidential Election

    @accalia said in Wherefore `from` in LINQ?:

    well for was already a language keyword, so can't use that or risk accidentally introducing ambiguity.

    I dunno. Seems like it could have been done contextually, or just based on the fact that, as far as I can tell, for (;;) requires parentheses and for x in y doesn't allow them. (Same goes for foreach - the parentheses appear to be required.)

    @accalia said in Wherefore `from` in LINQ?:

    as for select.... I think that's because they wanted the fluent style to flow simmilarly to the extension methods

    That one seemed reasonable to as part of the easy-to-read syntax.

    @accalia said in Wherefore `from` in LINQ?:

    or it could be the developer that did the demo used from and it stuck.
    demos can be like that.

    Yeah :/


  • Winner of the 2016 Presidential Election

    @RaceProUK said in Wherefore `from` in LINQ?:

    @accalia said in Wherefore `from` in LINQ?:

    well for was already a language keyword, so can't use that or risk accidentally introducing ambiguity.

    Only if you don't take context into account. It could have been done so that if for appears in the RHS of an expression, it's a LINQ for, else it's a loop for.

    I'm somewhat surprised this isn't valid:

    	var names = new List<string> { "accalia", "RaceProUK", "dkf" };
    	
    	from n in names
    	where n.Contains("a")
    	select names.Remove(n);
    

  • sockdevs

    @Dreikin said in Wherefore `from` in LINQ?:

    Seems like it could have been done contextually

    Contextually relevant keywords are hell for compilers.

    trust me on this, you do not want your compiler having to decide which operation you are asking for based on context, that way only madness lies.


  • sockdevs

    @Dreikin You can't do it like that anyway, as you can't mutate a collection being iterated. Instead, you should be doing this:

    var names = new List<string> { "accalia", "RaceProUK", "dkf" };
    names.RemoveAll(n => n.Contains("a"));
    

    Which, as it turns out, is 0% LINQ.


  • Winner of the 2016 Presidential Election

    @dkf said in Wherefore `from` in LINQ?:

    Have you ever done teaching of new users?

    My naive expectation is that the fact that it's pretty much the same thing in that section would make it easier to teach.

    foreach (x in y)
    {
        do stuff
    }
    

    doesn't seem all that conceptually different from

    foreach x in y
    where condition
    select projection
    

    even though the implementation is likely to be drastically different.

    Actually, given that, I don't see why they couldn't have simplified foreach syntax as well:

    foreach x in y
    {
        do stuff
    }
    

    and then you could combine them to conditionally do stuff:

    foreach x in y
    where condition
    {
        do stuff
    }
    

    or even

    foreach x in y
    where condition
    select projection
    {
        do stuff
    }
    

    From a student's perspective, they'd just be learning more ways to work with the same thing.


  • Winner of the 2016 Presidential Election

    @RaceProUK said in Wherefore `from` in LINQ?:

    @Dreikin You can't do it like that anyway, as you can't mutate a collection being iterated.

    D'oh! Yes, of course.


  • Winner of the 2016 Presidential Election

    @accalia said in Wherefore `from` in LINQ?:

    @Dreikin said in Wherefore `from` in LINQ?:

    Seems like it could have been done contextually

    Contextually relevant keywords are hell for compilers.

    trust me on this, you do not want your compiler having to decide which operation you are asking for based on context, that way only madness lies.

    I'd think that depends on the type of context, no? Some contexts are a lot clearer to figure out than others. E.g., one could view `\n' as a contextual keyword inside strings.

    But also, like I figured out while writing my twice-previous post, it doesn't really need to be (all that) contextual, but rather just an expansion on the allowable foreach syntax.


  • sockdevs

    @Dreikin said in Wherefore `from` in LINQ?:

    I'd think that depends on the type of context, no?

    tell you what. dake "COS-347 Compiler Architecture and Design" and ask that question again after the final. ;-)

    once you know how compilers actually work you'll understand why talk of contextual kewords make the people who know compilers shit their pants.

    @Dreikin said in Wherefore `from` in LINQ?:

    E.g., one could view `\n' as a contextual keyword inside strings.

    but it's not contextual.

    \n is "end of line" and strings are not allowed to cross lines, thus you get an error because the string was not terminated.


  • sockdevs

    @accalia said in Wherefore `from` in LINQ?:

    @Dreikin said in Wherefore `from` in LINQ?:

    Seems like it could have been done contextually

    Contextually relevant keywords are hell for compilers.

    trust me on this, you do not want your compiler having to decide which operation you are asking for based on context, that way only madness lies.

    And yet that's how the query comprehension syntax was implemented (yield return too IIRC). Then again, that was done to prevent breaking pre-LINQ code that uses those keywords as variables.



  • @Dreikin I think the difference lies in the return value. A for loop doesn't have a return value, while a from construct doesn't actually get evaluated until the iterator it returns is iterated or flattened. Or am I confusing concepts?



  • Here's what Eric Lippert (the C# and Haskell poobah) has to say about Linq syntax:


  • sockdevs

    @PleegWat No, you have it pretty much right:

    var q = from x in y select x; //This is where the query is built
    
    /*Much stuffs that doesn't use q*/
    
    foreach (var x in q) //This is where the query is executed
    {
        Frob(x);
    }
    

  • Winner of the 2016 Presidential Election

    @accalia said in Wherefore `from` in LINQ?:

    tell you what. dake "COS-347 Compiler Architecture and Design"

    I will once I can afford it.

    @accalia said in Wherefore `from` in LINQ?:

    and ask that question again after the final.
    once you know how compilers actually work you'll understand why talk of contextual kewords make the people who know compilers shit their pants.

    :shrug:

    • Compilers deal with difficult things all the time - that's their job, to make programming easier on humans.
    • That doesn't really address my assertion that the type of context matters to the difficulty of implementation.

    @accalia said in Wherefore `from` in LINQ?:

    but it's not contextual.
    \n is "end of line" and strings are not allowed to cross lines, thus you get an error because the string was not terminated.

    	var s = "Hello\nWorld!";
    	Console.WriteLine(s);
    

    Seems to work as expected. \n has special meaning inside the string that it doesn't outside of it.



  • @RaceProUK Indeed. Though as I think futher, you could terminate the from block with a do item that has a code block argument and aggregates the input iterator into void.


  • sockdevs

    @Dreikin said in Wherefore `from` in LINQ?:

    Seems to work as expected. \n has special meaning inside the string that it doesn't outside of it.

    \n is also syntactically invalid outside string literals.


  • Winner of the 2016 Presidential Election

    @PleegWat said in Wherefore `from` in LINQ?:

    @Dreikin I think the difference lies in the return value. A for loop doesn't have a return value, while a from construct doesn't actually get evaluated until the iterator it returns is iterated or flattened. Or am I confusing concepts?

    @RaceProUK said in Wherefore `from` in LINQ?:

    @PleegWat No, you have it pretty much right:

    var q = from x in y select x; //This is where the query is built
    
    /*Much stuffs that doesn't use q*/
    
    foreach (var x in q) //This is where the query is executed
    {
        Frob(x);
    }
    

    Yeah, and the version I wrote would map easily into that, using the same (or similar) process as mentioned in the article @Captain linked:

    foreach x in y
    where condition
    select projection
    {
        do stuff
    }
    

    would transform into

    foreach (var x in y.Where(n => condition).Select(n=> projection))
    {
        do stuff
    }
    

    which already works:

    	var names = new List<string> { "accalia", "RaceProUK", "dkf" };
    
    	foreach (var name in names.Where(n => n.Contains("a")).Select(n => n))
    	{
    		Console.WriteLine(name);
    	}
    

  • Winner of the 2016 Presidential Election

    @RaceProUK said in Wherefore `from` in LINQ?:

    @Dreikin said in Wherefore `from` in LINQ?:

    Seems to work as expected. \n has special meaning inside the string that it doesn't outside of it.

    \n is also syntactically invalid outside string literals.

    Y'know what, let me go look up "contextual keywords" again to make sure it means what I think it means before I did myself further into this particular hole.


  • Winner of the 2016 Presidential Election

    Okay, first look goes to

    A contextual keyword is used to provide a specific meaning in the code, but it is not a reserved word in C#. The following contextual keywords are introduced in this section:
    [list of contextual keywords such as var and get/set and await]

    All query keywords introduced in C# 3.0 are also contextual. For more information, see Query Keywords (LINQ).

    Which more-or-less says the argument is moot: they're already there, all the LINQ query keywords are contextual in the first place, and plenty of others that don't seem to case much trouble are as well.

    So given that, I'm arguing/asking about the particular keyword(s) chosen given it's already contextual, not whether it should be contextual.


  • Winner of the 2016 Presidential Election

    @Dreikin said in Wherefore `from` in LINQ?:

    @PleegWat said in Wherefore `from` in LINQ?:

    @Dreikin I think the difference lies in the return value. A for loop doesn't have a return value, while a from construct doesn't actually get evaluated until the iterator it returns is iterated or flattened. Or am I confusing concepts?

    @RaceProUK said in Wherefore `from` in LINQ?:

    @PleegWat No, you have it pretty much right:

    var q = from x in y select x; //This is where the query is built
    
    /*Much stuffs that doesn't use q*/
    
    foreach (var x in q) //This is where the query is executed
    {
        Frob(x);
    }
    

    Yeah, and the version I wrote would map easily into that, using the same (or similar) process as mentioned in the article @Captain linked:

    foreach x in y
    where condition
    select projection
    {
        do stuff
    }
    

    would transform into

    foreach (var x in y.Where(n => condition).Select(n=> projection))
    {
        do stuff
    }
    

    which already works:

    	var names = new List<string> { "accalia", "RaceProUK", "dkf" };
    
    	foreach (var name in names.Where(n => n.Contains("a")).Select(n => n))
    	{
    		Console.WriteLine(name);
    	}
    

    This also works already:

    	var names = new List<string> { "accalia", "RaceProUK", "dkf" };
    
    	foreach (var name in
    		from n in names
    		where n.Contains("a")
    		select n)
    	{
    		Console.WriteLine(name);
    	}
    

    but not this:

    	var names = new List<string> { "accalia", "RaceProUK", "dkf" };
    
    	foreach (var name in names
    		where n.Contains("a")
    		select n)
    	{
    		Console.WriteLine(name);
    	}
    

    And for this particular sub-topic, I'm saying it's a shame they didn't go just a little bit further to make that or my first example work. foreach x in y serves both purposes well, would reduce the contextual keyword count (foreach isn't contextual, while from is), and would usefully combine the two things in a natural manner.


  • sockdevs

    @Dreikin I wouldn't be surprised if Eric Lippert wrote a blog article about it at some point :slight_smile:

    Edit: I knew he'd have written something! :D


  • Winner of the 2016 Presidential Election

    More on contextual keywords, from Eric Lippert:

    One might wonder why on earth we added five contextual keywords to C# 1.0, when there was no chance of breaking backwards compatibility. Why not just make get set value add remove into “real” keywords?

    Because we could easily get away with making them contextual keywords, and it seemed likely that real people would want to name variables or methods things like get, set, value, add or remove. So we left them unreserved as a courtesy.



  • @Dreikin This works.

    foreach (var name in names
            .Where(n => n.Contains("a"))
            .Select(n => n))
    {
        Console.WriteLine(name);
    }
    

    And it is not much uglier.


  • Winner of the 2016 Presidential Election

    @Maciejasjmj said in Wherefore `from` in LINQ?:

    @Dreikin This works.

    foreach (var name in names
            .Where(n => n.Contains("a"))
            .Select(n => n))
    {
        Console.WriteLine(name);
    }
    

    And it is not much uglier.

    Yeah, but it also uses the method-call syntax, which already fits in as expected. The query syntax equivalent doesn't match the way I'd expect, though, which is my :confused: .

    Although on something else you raised earlier - preferring method syntax over query syntax - this seems relevant:

    tl;dr: query syntax has some optimizations built into the translation that you may not think to do when using the method syntax.



  • @Dreikin said in Wherefore `from` in LINQ?:

    query syntax has some optimizations built into the translation that you may not think to do when using the method syntax.

    I only skimmed through the article, but it seems like those optimizations aren't optimizations per se - instead of boosting your code's performance, they help the compiler choose the correct overload.

    And it's rather natural for me to pull the parent object into the SelectMany result. I certainly wouldn't write the nested lambda code - it's instinctively rather ugly.


  • Discourse touched me in a no-no place

    @Dreikin said in Wherefore `from` in LINQ?:

    query syntax has some optimizations built into the translation that you may not think to do when using the method syntax.

    It has much in common with how at least some scripting languages work. For all that the language in question is itself formally slow, it's quite fast for many key operations because of the ability to avoid doing things wrong and instead work more directly with the intent of the programmer. Once the runtime has information about intent, it can (probably) pick an efficient way to go about doing it.

    With C#, the key is not that the programmer couldn't do at least as good a job — with much thought and consideration — but rather that many people can get something better than they would have come up with by themselves given their constraints on knowledge and time.



  • @Dreikin

    and

    If you aren't careful you can do horrendous Linq queries that affect performance and readability.


  • Winner of the 2016 Presidential Election

    @lucas1 said in Wherefore `from` in LINQ?:

    @Dreikin

    and

    If you aren't careful you can do horrendous Linq queries that affect performance and readability.

    Results

    19.22 ns For-loop, string comparisons [ContainsLoop]
    54.60 ns Contains method [Contains]

    Huh. Did not expect that at all. My own testing shows a smaller gap:

    48.47 ns
    53.64 ns
    

    But still favors the for-loop version. Interesting.



  • @Dreikin It might have been written with an older compiler version that is optimising better.

    The real takeaway was using Any() which is a .NET 4.0 and up thing I believe.


  • Winner of the 2016 Presidential Election

    @lucas1 said in Wherefore `from` in LINQ?:

    @Dreikin It might have been written with an older compiler version that is optimising better.

    Running it again (everything the same) results in

    53.06 ns
    52.42 ns
    

    So yeah, it might be more-or-less caught up now from an earlier inefficient version.

    The real takeaway was using Any() which is a .NET 4.0 and up thing I believe.

    Okay.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.