The Inner Workings of Python Dataclasses Explained

[u/jpjacobpadilla]

Ever wondered how those magical dataclass decorators work? Wonder no more! In my latest article, I explain the core concepts behind them and then create a simple version from scratch! Check it out!

https://jacobpadilla.com/articles/python-dataclass-internals

(reposting since I had to fix a small error in the article)

[u/PurepointDog]

That felt so much hackier than I was expecting

[u/nekokattt → u/PurepointDog]

A fair bit of Python's standard library is like this. Look into collections.namedtuple for example.

If it isn't simple, it probably uses eval/exec/a lot of underlying stuff/C modules

[u/DuckDatum → u/PurepointDog]

Programming is like science. They teach you good guardrails, good rules of thumb, good yet often imprecise generalizations. Once you’re out there in the real world, dig into the weeds of things. You’re a better programmer when you know when to, and when not to, do things that are considered bad practice—like using eval.

[u/DigThatData → u/DuckDatum]

"bad practice" is a bit harsh, maybe "code smell"?

[u/zapman449 → u/DigThatData]

I trust very few people (mostly not including myself as well) to use eval reasonably. If I see that in a pull request the whole thing gets extra scrutiny.

[u/Skasch → u/zapman449]

I typically consider I use eval reasonably if I want to do something that doesn't seem possible without it, try a dozen alternatives, search for a few hours for different design patterns, sleep on it a few nights, ask a few colleagues their opinion, then write an apologetic comment above the line explaining why there's no way around it, then wrap that into a nice module so most other engineers won't have to think about it.

To be fair, I've never had to go that far.

[u/kuwisdelu → u/PurepointDog]

This is what you’re forced to do when your language doesn’t have lisp-like macros.

[u/DaelonSuzuka]

See also, the classic dataclasses talk by Raymond Hettinger:

https://www.youtube.com/watch?v=T-TwcmT6Rcw

[u/yrubooingmeimryte → u/DaelonSuzuka]

This is a great example of how NOT to do a tech talk. It takes him nearly 20 minutes to actually start talking about anything and even when he finally gets to the point he still constantly gets sidetracked talking about unrelated shit that just distracts from the the topic.

[u/victoriasecretagent → u/yrubooingmeimryte]

I typically enjoy his talks very much. Him and David Beazley.

[u/JanEric1]

Is there any specific reason that is done like that? I feel like one should be able to do this without exec, but I haven't put the implementations side by side to compare.

[u/FI_Stickie_Boi → u/JanEric1]

I believe the main reason is speed. attrs, the library dataclasses are based on, also do this, in order for the work to all be done during class creation, so that there's minimal overhead during "runtime" (ie. when you're instantiating classes, using methods, etc.) If you try and do this without eval/exec via decorators and all that, then you'll incur pretty significant runtime overhead because everytime you call a method, python will have to dig through multiple closures, which slows things down a lot.

[u/Awkward-Fisherman380]

That's Amazing. Very insightful Keep it up✌️🏼

[u/marcus-luck]

Great article! Thanks for writing and sharing!

[u/magnomagna]

However, if there are arguments in the decorator, the dataclass function will be called

Just a small nitpick... better be more specific:

However, if there are only keyword-only arguments in the decorator, the dataclass function will be called

[u/sohang-3112]

Good post!

[u/kuwisdelu]

Oh look it’s Greenspun’s tenth rule in action.