Nonstandard aesthetics #2555

clauswilke · 2018-05-04T21:17:31Z

I realized yesterday that with very little work I could address most of the issues I raised in #2345. So I would like to propose this pull request to substantially improve ggplot2's handling of nonstandard aesthetics. The pull request consists primarily of removing hardcoded aesthetics names in various places, and a few minor adjustments to handle the fallout. All the regression tests pass, and none of the changes should impact anybody who is using ggplot2 as they always have. Below follow some examples of what this code can do.

First, we can now specify the aesthetics independently of the scale name,
e.g., define the fill scale using scale_color_brewer():

library(ggplot2)
ggplot(iris, aes(x = Sepal.Length, fill = Species)) +
  geom_density(alpha = 0.7) +
  scale_colour_brewer(type = "qual", aesthetics = "fill")

Second, we can specify multiple aesthetics at once. The legends get merged if appropriate:

ggplot(iris, aes(x = Species, y = Sepal.Length, color = Species, fill = Species)) +
  geom_boxplot(alpha = 0.2) +
  geom_point(position = "jitter") +
  scale_colour_hue(c = 50, l = 40, aesthetics = c("colour", "fill"))

This also works if the aesthetics represent different data columns:

ggplot(iris, aes(x = Sepal.Length, y = Petal.Length, color = Sepal.Length, fill = Petal.Length)) +
  geom_point(shape = 21, size = 3, stroke = 2) +
  scale_colour_viridis_c(name = "Length",
                         option = "B", aesthetics = c("colour", "fill"))

The limits get merged automatically, as can be seen here more clearly:

df <- data.frame(x = c(1, 2, 3),
                 y = c(6, 5, 7))

ggplot(df, aes(x, y, color = x, fill = y)) +
  geom_point(shape = 21, size = 3, stroke = 2) +
  scale_colour_viridis_c(name = "value",
                         option = "B", aesthetics = c("colour", "fill"))

All of this also works with non-standard aesthetics, which I have implemented in ggridges.
First an example with guide_legend():

library(ggridges)
ggplot(iris, aes(x = Sepal.Length, y = Species, fill = Species, point_color = Species)) +
  geom_density_ridges(jittered_points = TRUE, position = "raincloud", alpha = .4) +
  scale_color_manual(values = c("#924658", "#366A17", "#126490"),
                     aesthetics = c("fill", "point_color"))

And now an example with guide_colorbar():

ggplot(iris, aes(x = Sepal.Length, y = Species, fill = ..x.., point_fill = Sepal.Length)) +
  geom_density_ridges_gradient(jittered_points = TRUE, position = "raincloud",
                               point_color = "black", point_shape = 21, scale = 1.1) +
  scale_color_viridis_c(
    option = "A",
    aesthetics = c("fill", "point_fill"),
    guide = guide_colorbar(available_aes = c("fill", "point_fill")),
    name = "Sepal Length"
  )

clauswilke · 2018-05-04T21:25:15Z

Two additional comments:

I didn't add any regression tests, visual or otherwise. I'm happy to do so if otherwise this patch looks acceptable.
There is one breaking change, in that guide_train() now requires an additional argument. This should only be a problem for code that defines custom guides, and I'm not aware of any code that does that. It's certainly not something people commonly do.

thomasp85 · 2018-05-04T21:31:45Z

I do in ggraph but that shouldn’t weight against this PR

clauswilke · 2018-05-04T21:36:59Z

@thomasp85 Also, the fix takes a minute. You simply need to use the provided aesthetic instead of the first one from scale$aesthetics: https://github.com/clauswilke/ggplot2/blob/3841ddaee3d549e15df520cf59c89cbcc4565770/R/guide-legend.r#L225
I needed to do this because we also want to build guides for the second or third aesthetic listed for a scale.

Would you mind installing this PR and testing if it causes any problems with ggraph?

thomasp85 · 2018-05-04T22:28:11Z

sure, but not before next week. I’ll get back to you

hadley · 2018-05-07T14:01:33Z

I really dislike scale_colour_brewer(type = "qual", aesthetics = "fill") because we have an argument value that conflicts with the function name. The implementation seems simple enough to be worth doing (I haven't deeply reviewed yet, but it seems reasonable), but I don't like the weirdness this introduces into the API. What could we do to avoid it?

thomasp85 · 2018-05-07T14:11:27Z

Maybe have an aesthetic-less scale available where the aesthetic can be set in the function call, so you’d do:

p + scale_brewer(..., aes = c(“fill”, “colour”))

It would require a huge amount of new exported functions, but api wise I think it is the most pretty approach

clauswilke · 2018-05-07T14:25:58Z

I previously commented on the naming issue here: #2345 (comment)

Words such as "color", "shape", "size" etc serve two purposes in the current ggplot2. On the one hand, they identify the aesthetic. On the other hand, they indicate the type of scale (color, shape, size, etc). If we only focus on the latter for a moment, it's clear that scale_colour_* is a color scale, scale_size_* is a size scale, scale_shape_* is a shape scale, etc. And I believe this is on some level the more intuitive perspective, given how often people on stackoverflow try to use scale_colour_* to style a fill aesthetic. And also, as I said in my earlier comment, the generic version of scale_shape() would be scale(), which doesn't make sense.

For these reasons, I'm very comfortable with scale_colour_brewer(..., aesthetics = "fill"). If ggplot2 had been written like this from the beginning, nobody would have batted an eye.

clauswilke · 2018-05-07T14:36:27Z

Another way to think about it (which I'm proposing more as a thought experiment): Remove scale_fill_*(), and you'll end up with a logical and consistent interface. scale_colour_*() / scale_fill_*() is the only scenario in standard ggplot2 where you have two scales that do the same thing (map to color) but have different names because they handle different aesthetics.

hadley · 2018-05-07T16:00:05Z

Ok, that seems like a compelling argument.

We could also (eventually) consider making scale_colour_blah() have aesthetic = c("colour", "fill") — that would require more effort if you wanted to map fill and colour aesthetics to different scales, but that seems totally reasonable.

clauswilke · 2018-05-09T06:51:10Z

I have rebased and added a few regression tests.

hadley · 2018-05-09T15:26:59Z

Can you have a look at why travis is failing please?

clauswilke · 2018-05-09T16:21:08Z

Not sure what the problem is. CRAN check works for me. I don't seem to be able to see the travis build log for this repository, and that makes figuring these things out more difficult. The one thing I noticed was an out-of-date .Rd file, so maybe that was it. I have committed it now.

karawoo · 2018-05-09T16:28:16Z

It looks like the Travis build timed out during installation of the dependencies.

clauswilke · 2018-05-09T16:59:04Z

Either way it succeeded now. And I finally figured out where I can see the build log, so I'll be more informed in the future.

can make full use of multiple aesthetics for one scale.

hadley

I think if we back off the user facing changes just slightly this is good to merge.

hadley · 2018-05-09T20:58:23Z

NEWS.md

@@ -211,6 +215,10 @@ up correct aspect ratio, and draws a graticule.

 * Legends no longer try and use set aesthetics that are not length one (#1932).

+* All scales that are not position scales now have an `aesthetics` argument
+  that can be used to set the aesthetics the scale works with, regardles of
+  the name of the scale function. (@clauswilke)


For this version, maybe we could just add that argument to colour scales? You'll still be able to set it via ... but it won't be so in your face.

The problem is that you can't set the aesthetics via ..., that's what brought me down this rabbit hole in the first place. The reason is that aesthetics is hard-coded as an argument to discrete_scale() or continuous_scale() in all scale functions.

As an example, let's look at scale_colour_brewer(), which is currently defined as follows:

scale_colour_brewer <- function(..., type = "seq", palette = 1, direction = 1) { discrete_scale("colour", "brewer", brewer_pal(type, palette, direction), ...) }

If we try to call it with a custom aesthetic, the following happens:

ggplot(iris, aes(Sepal.Length, fill = Species)) + geom_density() + scale_colour_brewer(aesthetics = "fill") # Error in self$palette(n) : attempt to apply non-function

Since we provide an aesthetics argument, it is handed off to discrete_scale() via ..., but now all the other arguments are shifted by one and so the palette function gets assigned to the wrong argument.

Here is a very simplified example to demonstrate the issue:

f <- function(a, b, c) { print(a) print(b) print(c) } g <- function(...) { f(4, 5, ...) } g(1) # [1] 4 # [1] 5 # [1] 1 g(a = 1) # [1] 1 # [1] 4 # [1] 5

I could process the ellipsis with match.call() and then call the lower-level scale function with do.call(), but that would be a lot of extra code in every single scale function, and it'd be not very readable. I can ponder some more whether there's some other way to make this happen, but I haven't come up with one yet.

Is there a way to test whether the ellipsis contains a specific argument, without actually evaluating the ellipsis? Maybe some rlang feature I'm not aware of?

Ugh, of course, blech. What we just leave off the non-colour aesthetics for now and consider the user API in the future? I'd like to get this into this release, but I don't have the time to think through any big API changes.

In that case, how do you feel about exporting manual_scale() (possibly under a different name, e.g. scale_discrete_manual())?

ggplot2/R/scale-manual.r

Line 74 in 0dab044

manual_scale <- function(aesthetic, values, ...) {

The problem is that there is currently literally no simple way of defining, for example, a discrete shape scale for an aesthetic other than shape. One always has to reimplement manual_scale(), and it's the reason I invented scale_discrete_manual() in ggridges:
https://github.com/clauswilke/ggridges/blob/4e94e9a32b976c5a2eedc6fa04b104178f3e9767/R/scale-.R#L26

Yeah, that sounds good, and I think that name is fine.

Ok, to summarize, I will:

Revert addition of aesthetics argument for non-color scales.

Add and export scale_discrete_manual() which works just like manual_scale().

I'll try to get this done by Friday.

…e manual scale.

clauswilke · 2018-05-10T03:49:12Z

I have made these changes now. The aesthetics argument remains only for scales that set a color. Throughout the documentation, I have explained this as an option for setting colour and fill aesthetics at the same time, so it shouldn't be too confusing.

I also added three new generic scales that I think are needed, scale_discrete_manual(), scale_discrete_identity(), and scale_continuous_identity().

hadley · 2018-05-10T13:38:24Z

Thanks!

lock · 2018-11-17T20:46:52Z

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/

clauswilke force-pushed the nonstandard-aesthetics branch from 3841dda to 1f09a31 Compare May 9, 2018 02:00

clauswilke added 5 commits May 9, 2018 15:56

enable guide_colorbar to work with nonstandard aesthetics

3ce823d

remove hardcoded aesthetics setting for scales; adjust guides so they

3794036

can make full use of multiple aesthetics for one scale.

update NEWS

882eedb

add regression tests

9628730

update documentation

27788e1

clauswilke force-pushed the nonstandard-aesthetics branch from 92a43a0 to 27788e1 Compare May 9, 2018 20:57

hadley reviewed May 9, 2018

View reviewed changes

clauswilke added 5 commits May 9, 2018 21:21

revert aesthetics argument for non-color scales

a3d16cc

revert aesthetics argument for non-color scales, add exported discret…

0a48c5c

…e manual scale.

add generic manual and identity scales

a49f59b

add test cases for new generic scales.

344a690

improve docs

a78d22a

hadley merged commit 7ac9e42 into tidyverse:master May 10, 2018

This was referenced May 10, 2018

Better support for custom color and fill aesthetics? #2345

Closed

unused argument (output) #2597

Closed

clauswilke deleted the nonstandard-aesthetics branch May 21, 2018 20:29

lock bot locked and limited conversation to collaborators Nov 17, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nonstandard aesthetics #2555

Nonstandard aesthetics #2555

clauswilke commented May 4, 2018 •

edited

clauswilke commented May 4, 2018

thomasp85 commented May 4, 2018

clauswilke commented May 4, 2018

thomasp85 commented May 4, 2018

hadley commented May 7, 2018

thomasp85 commented May 7, 2018

clauswilke commented May 7, 2018

clauswilke commented May 7, 2018

hadley commented May 7, 2018

clauswilke commented May 9, 2018

hadley commented May 9, 2018

clauswilke commented May 9, 2018

karawoo commented May 9, 2018 •

edited

clauswilke commented May 9, 2018

hadley left a comment

hadley May 9, 2018

clauswilke May 9, 2018

hadley May 9, 2018

clauswilke May 9, 2018

hadley May 9, 2018

clauswilke May 9, 2018

clauswilke commented May 10, 2018

hadley commented May 10, 2018

lock bot commented Nov 17, 2018

Nonstandard aesthetics #2555

Nonstandard aesthetics #2555

Conversation

clauswilke commented May 4, 2018 • edited

clauswilke commented May 4, 2018

thomasp85 commented May 4, 2018

clauswilke commented May 4, 2018

thomasp85 commented May 4, 2018

hadley commented May 7, 2018

thomasp85 commented May 7, 2018

clauswilke commented May 7, 2018

clauswilke commented May 7, 2018

hadley commented May 7, 2018

clauswilke commented May 9, 2018

hadley commented May 9, 2018

clauswilke commented May 9, 2018

karawoo commented May 9, 2018 • edited

clauswilke commented May 9, 2018

hadley left a comment

Choose a reason for hiding this comment

hadley May 9, 2018

Choose a reason for hiding this comment

clauswilke May 9, 2018

Choose a reason for hiding this comment

hadley May 9, 2018

Choose a reason for hiding this comment

clauswilke May 9, 2018

Choose a reason for hiding this comment

hadley May 9, 2018

Choose a reason for hiding this comment

clauswilke May 9, 2018

Choose a reason for hiding this comment

clauswilke commented May 10, 2018

hadley commented May 10, 2018

lock bot commented Nov 17, 2018

clauswilke commented May 4, 2018 •

edited

karawoo commented May 9, 2018 •

edited