Does something feel off about Matplotlib’s API to you? If you think Matplotlib is harder to use than it needs to be, your intuition is correct.
If you think the reason why Matplotlib has a cumbersome API is because it has so much going on under the hood that it needs to be complicated, you are incorrect.
The Problem
I want to emphasize that Matplotlib’s bad API is of little fault to its core developers or original visionary, John Hunter. Or perhaps it’s more accurate to say that it’s hard to blame them for their decisions. Matplotlib’s first release was in 2003, and Python was still a somewhat obscure language back then. The idea of “Pythonic” code did not really exist in 2003. People were still coming to Python from other languages, such as Java, and these very skilled developers coded in Python as if it was Java. PEP 8 was only 2 years old at that point, and it was really the only guide to designing Python code (to the credit of Matplotlib’s early developers, the API does adhere to the PEP 8 style guide). Raymond Hettinger had only been a core Python developer for 3 years.
The developers of Matplotlib made the decision to emulate MATLAB’s plotting API with the pyplot module, so that MATLAB users would feel more at home. This is a perfectly reasonable design decision when you are up against a competing product (MATLAB in 2003) and yours is the lesser used product or is on a lesser used platform (Python, broadly, in 2003). Of course, today Python is the more used product.
These days, it is preferable to interact with Matplotlib’s “figure” and “axis” objects directly, using fig, ax = plt.subplots(), and ditch the pyplot interface. Matplotlib’s examples documentation does not include any examples that build graphs exclusively on the pyplot MATLAB-esque API, as Matplotlib was originally envisioned.
So they’ve fixed the issue now, no? Unfortunately not. There are two awkward issues, one of which was totally unavoidable in that transition. The unavoidable problem is now there are two Matplotlib APIs, which means if you search on for StackOverflow for answers to your Matplotlib questions, it’s a crapshoot on whether someone will provide an answer that uses your preferred API.
The second issue is that the classes use Java-esque “setter” methods, such as ax.set_xlim(). As far as I can tell, Python’s @property decorator and its corresponding setter attribute existed back in 2003, so it seems this was theoretically avoidable even at the time. But again: the idea of code being “Pythonic” did not exist back when Matplotlib’s API was being fleshed out. Python did not have its own idiom. It’s really hard to fault the developers for following the more Java-esque convention.
So if Matplotlib’s API is so unidiomatic, why haven’t they fixed it yet? Because once an API is set in stone, you can’t change it without breaking other people’s code. And breaking other people’s code is bad– far worse than a bad API. Instead of “fixing” Matplotlib directly, it’s preferable to build new products. And others have already done that with tools such as Seaborn and Plotly. Both of these libraries have more idiomatic APIs that avoid global state and set_* methods. [See footnote 1.]
Consider Matplotlib to be a lesson on the importance of getting your API right the first time. Most things can be fixed later, but bad APIs, along with bad input/output specifications, are forever. Once you have added something to your API and the number of people using your API is N>1 (i.e. more than just you), it is no longer deletable.
Recreating Matplotlib’s API Style
It’s not too hard to recreate Matplotlib’s API design in another context, explore why it’s bad, explore how we would make it better, and discuss how to avoid the things that make its API bad.
All the code I’ve written for this post is available on Github. As a forewarning, Github’s Jupyter rendering does not render some of the outputs properly, but you’ll be able to see it on Jupyter natively.
https://github.com/ryxcommar/applelib_demo
I also want to emphasize that this is a demo on how not to design APIs, to the extent that you can help it. So please do not emulate what you see in this section. In the past, I’ve written cursed code for the forces of evil. Today. I am writing cursed code for the forces of good, but it’s only good if everyone learns the intended lessons from this.
Let’s say instead of charts and graphs, we just wanted to create “Apple” objects and print them out in a Jupyter notebook. So we design a new and cool library called “Applelib,” directly inspired by Matplotlib. For some weird reason, let’s say we also want to be faithful to Matplotlib’s API. How can we go about doing this?
The Apple class
The following code defines an Apple class. An Apple can have a color attribute, and the color defines how it gets printed in Jupyter notebooks. The special print functionality is handled by the _repr_html_ method, which Jupyter handles automatically (no additional fuss required).
# Contents of apples.py
from .config import set_current_apple
# Note: `valid_color` just returns True or False if it's a valid color.
# It's in the full GitHub code, but not the blog post.
from .utils import valid_color
APPLE_TEXT = [
' _ ',
' / ',
' #### ',
'######',
'######',
' #### '
]
DEFAULT_APPLE_COLOR = 'red'
class Apple(object):
_color = DEFAULT_APPLE_COLOR
def __init__(self, color: str = None):
self._color: str = color or self._color
set_current_apple(self) # NOTE: We'll get back to this later.
def set_color(self, color: str) -> None:
if valid_color(color):
self._color = color
else:
raise ValueError(
f'{color.__repr__()} is not a valid color.'
)
@property
def color(self) -> str:
return self._color
def __repr__(self) -> str:
return '\n'.join(APPLE_TEXT)
def _repr_html_(self) -> str:
"""Special repr for Jupyter notebooks."""
txt = '<br>'.join(APPLE_TEXT).replace(' ', ' ')
return f'<font color="{self.color}"><tt>{txt}</tt></font>'
As far as I can tell, the following code should work so long as I also import the Apple class into Applelib’s __init__.py:
import applelib as apl
granny_smith = apl.Apple()
granny_smith.set_color('green')
granny_smith
It works, but is it a bad API? Yes, because set_* methods are unidiomatic in Python. The way you should set things in Python is with the equals sign, =.
granny_smith.color = 'green'
If a simple equivalence doesn’t cut it (it usually does though), there are ways to make it more complicated that we’ll discuss momentarily.
Pieplot API
Get it? “Pie” plot? As in apple pie? … Never mind.
For the Pieplot API, we want the following code to work by printing out a blue apple. For the sake of simplicitly, I’m only designing plt.show() to work in Jupyter notebooks. We also aren’t going to cheat: plt will be a module the same way matplotlib.pyplot is, not some instance of a class with a color property and a show() method.
import applelib.pieplot as plt
plt.color('blue')
plt.show()
The way Matplotlib’s Pyplot API works is that it makes a bunch of calls to two functions: gca() and gcf(). “gca” refers to “get current axis,” and “gcf” refers to “get current figure.” These functions take whatever the “current” axis or figure is, and modifies it that way. What’s current? Whatever fig or ax the global state says is current!
We can recreate a simpler version of the gca and gcf functionality in a file we’ll call config.py:
# Contents of config.py
_current_apple = None
def get_current_apple() -> 'Apple':
"""Get the current apple from the global state."""
if _current_apple is None:
# Lazy load to avoid circular dependency
from .apples import Apple
set_current_apple(Apple())
return _current_apple
def set_current_apple(apple: object) -> None:
globals()['_current_apple'] = apple
Then we can build the Pieplot API on top of config.py:
# Contents of config.py
from .config import get_current_apple
from copy import copy
def color(color: str) -> None:
return get_current_apple().set_color(color)
def show():
return copy(get_current_apple())
Is this code bad? In context, yes. The sin that’s being committed here is the use of the module level global state to retrieve and store an object that we intend on mutating. Bonus badness points because get_current_object() is not even guaranteed to reference the same object when you call it twice!
“When will that ever happen?” you ask. Often the answer to that is “a race condition,” but in this shoddy cursed library, the answer is much simpler.
Remember this in apples.py?
set_current_apple(self) # NOTE: We'll get back to this later.
That’s right. If you are modifying an Apple in the global state, then create a new Apple object, you’ll set the new one to be the “current” Apple and lose the original Apple and all of its mutations.
“Just get rid of the line of code in Apple.__init__ that says set_current_apple(self), then,” you say. “Or make it so that creation of a new Apple object only sets it to be the ‘current Apple’ if a ‘current Apple’ doesn’t already exist,” you add.
All fair suggestions, but none of those choices necessarily make sense either! In the following code, why should the old Apple be the “current” Apple, if I just created an Apple after it? What’s “current” about the first apple?
macintosh = get_current_apple()
fuji = Apple()
get_current_apple() # Should this be fuji, or macintosh?
When I ask, “Should this be fuji or macintosh?” I’m not asking what will happen, but what should happen, as in: what behavior should we decide on? The fact that we have to decide between two radically divergent behaviors for our simple Applelib library, neither of which is obviously correct or preferable, should be a hint that we’re possibly doing something wrong. And that thing is messing around in the global state.
Is code that mutates the global state always bad? Not necessarily, even though you should avoid it to the extent that you can, and you should feel a little guilt and shame before you type out code that is intended on allowing you to mutate objects in the global state. The most typical example of acceptable global state mutations are cosmetic package level options. For example, Pandas lets you type the follow code:
import pandas as pd
pd.set_option('display.max_rows', 100)
This code commits two sins: it mutates something in the global state AND it uses a Java-esque setter function to do so. Despite that, it’s actually totally fine! pd.set_option is a fine use of mutation of global variables. How you display a DataFrame is something that can be shared across all DataFrames you create, and is more convenient to do at the start of a Jupyter notebook (i.e. the sort of environment where display options matter more) rather than inside of a cell after you imported a CSV file or whatever. Additionally, because the option is cosmetic, even if it does get modified in weird ways as your code runs, it’s likely not going to cause any harm.
There are a small number of options in Pandas that are not cosmetic, such as io.hdf.dropna_table. These are a little more dangerous to touch. But for those options, you can use pd.option_context, a clever little context manager function that temporarily sets an option, then reverts it back when it exits. Temporarily modifying the behavior of things inside of a context decorator is definitely fair game.
Fixing Our API: The Simple Solution
Unlike Matplotlib, we don’t have hundreds of thousands of people using our API, so we won’t break anyone’s code by fixing it. Let’s fix it before we publicly release it and people start using our code!
As noted, the simplest and easiest way to fix our API is to just let color be a regular `ol attribute.
# pulled from ./applelib/experimental/v1.py
class NewApple(object):
color = 'red' # Default color is red
def __init__(self, color: str = None):
self.color: str = color or self.color
def __repr__(self) -> str:
return '\n'.join(APPLE_TEXT)
def _repr_html_(self) -> str:
"""
Special repr for Jupyter notebooks.
"""
txt = '<br>'.join(APPLE_TEXT).replace(' ', ' ')
return f'<font color="{self.color}"><tt>{txt}</tt></font>'
In this case, we set the color the way we set any variable, i.e. with an equals sign No set_color in sight, and certainly no plt.color()‘s either. And doing so makes the color attribute exactly equivalent to what was passed on the right hand side:
cripps_apple = NewApple()
cripps_apple.color = 'pink'
Last but not least, we are discarding with plt.show() in the context of Jupyter notebooks. Printing the object will be as simple as print(cripps_apple), or in a Jupyter notebook. simply cripps_apple at the bottom of the cell.
Do the above approaches (set with an equals sign, or when initializing the class) feel more natural to you? If you write a lot of Python code, they should!
Fixing Our API: Slightly More Complicated Solution
“Wait a minute!” you exclaim. “We lost some functionality in this new Apple class. It no longer checks to see whether we set the color attribute to be a valid color.”
That is true. There is a fix to that, but before I present it, I want to argue that I do not think this was actually very important functionality to begin with! It cluttered our code but provided little benefit. Ask yourself:
- How bad is it when an error happens due to an invalid color? If we pass an invalid color, such as
color = 'asdf', it’s unlikely we cause a complete catastrophe that would cost hundreds or thousands of dollars. - How many users does it affect? As far as we can tell, you and I are the only users of this library.
- Can invalid inputs be used maliciously? As far as I can tell, no, you’ll just have weird looking apples.
- Do users eventually get feedback when they’ve passed an invalid color? Yes– their apples will look wrong when they print them.
In a real world context, you should take these considerations into account before cluttering your code with stuff like input validation or input transformation. That said, if we want to be faithful to the original API, we can implement that. We can use the @<property>.setter design to do that:
class AltNewApple(object):
_color = 'red' # Default color is red
def __init__(self, color: str = None):
self._color: str = color or self._color
@property
def color(self) -> str:
return self._color
@color.setter
def color(self, color: str) -> None:
if valid_color(color):
self._color = color
else:
raise ValueError(
f'{color.__repr__()} is not a valid color.'
)
def __repr__(self) -> str:
return '\n'.join(APPLE_TEXT)
def _repr_html_(self) -> str:
"""
Special repr for Jupyter notebooks.
"""
txt = '<br>'.join(APPLE_TEXT).replace(' ', ' ')
return f'<font color="{self.color}"><tt>{txt}</tt></font>'
The property setter decorator also lets you do something that’s very important for big, complex libraries like Matplotlib. Let’s say your AppleLib project became a giant monstrosity, and referring to colors as string variables simply isn’t cutting it anymore. So you implement a Color class. How do you let users input string objects as colors, while still always having the _color attribute be a Color object? Simple:
@color.setter
def color(self, color: Color) -> None:
if not isinstance(color, Color): # This allows for string inputs
color = Color(color)
self._color = color
Presumably, the Color.__init__ method is where the input validation is; it’s no longer necessary in your setter. No need to validate twice.
Remember to adhere to the Spider-Man rule when it comes to setters: “with great power comes great responsibility.” When someone sets an attribute, they typically expect that the thing they set is exactly the same thing they’ll see when they try to get it later. Because of that principle and other things above, it’s a pretty rare occasion when you find an explicit property setter in my code. Simply assigning attributes with the equals sign the naive way usually does the trick. Also, most people aren’t designing libraries so complex and deep in the weeds on the back-end, yet simple/stupid on the front-end, that they’ll require extensive use of typecasting setters. Pretending your code is more important than it actually is often makes your code worse, not better.
Pandas is an example where typecasting setters are used extensively, and it makes the library a lot better. Typically, something like my_object.some_property = 4 turns some_property into an int equal to 4. But for a Pandas DataFrame, df['some_column'] = 4 turns df['some_column'] into a pd.Series object with a bunch of 4’s, even though you technically set it to be an integer of 4. Pandas’s backend is extraordinarily complicated, but the basic idea behind what’s happening is not dissimilar to the example above: it sees that you’re trying to set with an int, and Pandas says “wait a sec, we don’t want this to be an int, we want it to be a pd.Series instead.”
Bonus Section: Dispatch Functions
Is there an alternative approach to using isinstance(color, Color), as shown in the last example?
Depending on how complicated your input is, you can use functools.singledispatch to call different functions, that will vary depending on the data type of the first positional arg. The below code only works in Python 3.7, both due to the dataclasses module import and due to using type hints for registering the dispatchers. The below code also doesn’t do justice showing off how valuable dispatch functions can be, but it should be pretty readable:
from dataclasses import dataclass # requires Python 3.7+
from functools import singledispatch # requires Python 3.3+
@dataclass
class Color:
color: str
@singledispatch
def _parse_color(val: str) -> Color:
# If it's not yet a Color, convert to a Color first.
return Color(val)
@_parse_color.register
def _(val: Color) -> Color:
# If it's already a Color, return as-is.
return val
class Apple(object):
_color = None
@property
def color(self) -> Color:
return self._color
@color.setter
def color(self, val):
self._color = self._parse_color(val)
if __name__ == '__main__':
granny_smith = Apple()
granny_smith.color = 'green'
# The setter should have transformed the str to a Color.
assert isinstance(granny_smith.color, Color)
Can we do better than that? As of writing, no. In the future, maybe!
functools.singledispatchmethod (requires Python 3.8+) is a decorator that works similarly to singledispatch, but for methods. According to the documentation:
@singledispatchmethodsupports nesting with other decorators such as@classmethod. Note that to allow fordispatcher.register,singledispatchmethodmust be the outer most decorator. […] The same pattern can be used for other similar decorators:staticmethod,abstractmethod, and others.
Unfortunately, “others” doesn’t include property(method).setter, as far as my experimentation leads me to believe.
If you try it out yourself, you’ll find that .color turns into a regular method, not a property. One clever trick would be to create a single dispatch method called _parse_color and then assign it as a setter via color = color.setter(_parse_color), but… that doesn’t work either. You’ll get a TypeError: 'singledispatchmethod' object is not callable.
Bonus Section: Code for the header image
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
np.random.seed(42069)
# Create dataframe
# ~~~~~~~~~~~~~~~~
df = pd.DataFrame([], index=range(150))
df['x'] = df.index + 1
df['y'] = np.random.normal(0, 1, size=len(df)).cumsum()
# Create plot
# ~~~~~~~~~~~
subplot = sns.lineplot(x='x', y='y', data=df)
subplot.figure.set_dpi(500)
subplot.axes.get_lines()[0].set_color('#DDDDDD')
for row in df.itertuples():
subplot.axes.text(x=row.x, y=row.y, s='😱')
subplot.figure.savefig('./aaaaaaaah.png')
Bonus Section: Footnote 1 re: Seaborn
Seaborn, notably, is built on Matplotlib and is effectively a higher-level wrapper that takes the good parts about Matplotlib (its smoothly rendered graphics) and makes the interface more idiomatic, at least at the Seaborn module level. Objects returned by Seaborn’s API tend to be Matplotlib Figure objects, so once you’ve built the figure with a 1-liner from Seaborn’s API, you’re back to square 1 with Matlab.
Building wrappers on top of more base-level libraries to make coding more boilerplate is very common: take for example the very popular requests library, which is built on urllib3. The requests library effectively hides stuff users like connection management and reduces HTTP requests down to their boilerplate.
If you think I’m opinionated about APIs, you should check out the README for Requests’s 2.7 release. Whoever wrote this is a bit smug about how user-friendly Requests is:
Requests is an Apache2 Licensed HTTP library, written in Python, for human beings.
Most existing Python modules for sending HTTP requests are extremely verbose and cumbersome. Python’s builtin urllib2 module provides most of the HTTP capabilities you should need, but the api is thoroughly broken. It requires an enormous amount of work (even method overrides) to perform the simplest of tasks.
Things shouldn’t be this way. Not in Python.
I mean, I can’t say they’re wrong…
r y x, r 
You must be logged in to post a comment.