Anonymous python objects?

stillwater

I have a method in the data access layer with a bunch of objects I am trying to bring to the view files(acts as the contoller) in Django. The objects in the DAL method are a bunch of s3 objects with details such as name, last accessed, size and a couple other attributes.

If this was LINQ, at the end of the query we'd have something like

return new { name = s3Object.name, last_accessed = s3Object.last_acc ... }

which can be returned from the method.

Is there an equivalent in python? As far as I can tell there does not seem to be any straight forward way to create anonymous python objects like so.

Right now I am returning a list of tuples with n attributes each to the controller. Something like

def return_s3_stuff():
    my_list = [ ]
    for obj in s3objects:
        list.append((obj.name, obj.last_accessed, obj.size..))
    return my_list

I do not want to create a class in the same file just to do this. Also the reason I'm thinking about using an anon object is that using a tuple inside lists feels like a hack.

I want this to be strongly typed and given how there is no compile time checking here, I'm worried somewhere down the line any change here like adding an attribute to the tuple here or changing the order for whatever reason might break the unpacked tuple elsewhere.

The closest slightly better solution i can find is using a namedtuple inside the list.

I'm not an expert in python by any means and If I'm doing anything wrong apart from the issue at hand, please feel free to point me in the right direction.

Edit: Fixed formatting.

dkf

@stillwater Your main option is a namedtuple, which is at least an immutable object so there's not much trouble you can get into. (You'll probably want to give it a name for your own convenience, but that's just because truly unnamed things have a tendency to be a bit annoying. The name can be just for your implementation code though; prefix with _ to discourage others.) If you need to give the object methods, you'll instead want to specify a class with __slots__ on it, which again stops most of the worst shenanigans.

And regularly run a tool like pylint over your code. It finds all sorts of things early.

stillwater

@dkf Slots seem like a swell idea. If I'm feeling particularly squeamish using python, I'll probably use slots.

Thanks. namedtuple looks like it would fit the problem.

Gąska

You could also use a dictionary.

dkf

@Gąska Depends on whether you want the flexibility, or whether you want to lock things down a bit more.

Gąska

@dkf if he wanted to lock things down, he'd be using a statically typed language.

stillwater

@Gąska said in Anonymous python objects?:

@dkf if he wanted to lock things down, he'd be using a statically typed language.

I do want to lock things down but I cannot use a statically typed language due to raisins.

stillwater

@dkf said in Anonymous python objects?:

@Gąska Depends on whether you want the flexibility, or whether you want to lock things down a bit more.

The namedtuple is in place already and the code is clearer. I think I've reached the point where I can offload this to someone else.

dkf

@stillwater said in Anonymous python objects?:

I think I've reached the point where I can offload this to someone else.

That's a win!

Gąska

@stillwater said in Anonymous python objects?:

@Gąska said in Anonymous python objects?:

@dkf if he wanted to lock things down, he'd be using a statically typed language.

I do want to lock things down but I cannot use a statically typed language due to raisins.

The thing is, you can't really lock down things in dynamic languages. Using namedtuple doesn't bring you any benefit on this front. Field accesses are only resolved at runtime, the moment it's being used. Whether you used dictionary indexing or tuple member access, the end result is the same - invalid name won't be caught until you run the code that uses invalid name, and the only difference will be in exception type.

It does make for a nicer API in some cases, though. That's definitely a reason to use it instead of dictionary. But other times, dictionary is nicer. It's all up to personal preference. Functionally, the two are equivalent. One might say namedtuple has slightly better performance, but it's Python - performance is already awful anyway, and these micro-improvements don't matter.

dkf

@Gąska said in Anonymous python objects?:

Using namedtuple doesn't bring you any benefit on this front.

You are wrong. It prevents some other code from poking extra fields into the object. Yes, some “programmers” like to do that.

stillwater

@dkf said in Anonymous python objects?:

That's a win!

Everytime I'm thrown into the dynamically typed languages land, I feel like I'm putting my dick in a blender with a time bomb. The entire thing could have been done far smoother in anything statically typed. Half the time was spent correcting typos.

Gąska

@dkf said in Anonymous python objects?:

@Gąska said in Anonymous python objects?:

Using namedtuple doesn't bring you any benefit on this front.

You are wrong. It prevents some other code from poking extra fields into the object. Yes, some “programmers” like to do that.

Huh, I wasn't aware of that. Regular Python classes monkey-patch just fine. namedtuple must be using some deep Python magic to raise an error. I looked at the source, and I'm completely stumped on this one - I see nothing about checking accessed field names, nor any magic definition that would look like it's invoking this behavior.

dkf

@Gąska said in Anonymous python objects?:

namedtuple must be using some deep Python magic to raise an error

The magic is that it inherits from tuple. The namedtupleness is that it allows you to access the tuple fields by name.

Gąska

I think I figured it out. namedtuple generates __slots__ attribute, and if a class is defined with __slots__ attribute, the interpreter doesn't use dictionary to store instance's data, but rather a fixed size memory. This has a side effect of preventing additional fields from being added. Modifying this attribute for an instance, either adding or removing it, has no effect. From what I've read, __slots__ is supposed to hold field names, but namedtuple uses empty tuple, and it works just fine. Go figure. Furthermore, the generated class uses properties for the declared fields, and it lacks setter and deleter, making the objects immutable. Interestingly, I couldn't find any mechanism that would let you bypass those accessor methods - unlike any other dynamically typed language I know. Also, it seems the Python community considers properties to be harmful. I haven't read up the reasons why, but quite a few blog posts with very suggestive titles have popped up while I was googling for all these other things.

Moral of the story: add __slots__ = () to every single Python class you ever write.