The Gender of Big Data

When I grow up I want to be a dancer (Carmen, my beautiful daughter)

The presence of women in positions of responsibility inside Big Data companies is quite far of parity: while approximately 5o% of world population are women, only 7% of CEOs of Top 100 Big Data Companies are.

To do this experiment, I did some webscraping to download the list of big data companies from here. I also used a very interesting package called genderizeR, which makes gender prediction based on first names (more info here).

Here you have the code:

paste0("", 1:45) %>%
c(., paste0("",1:30)) %>%
c(., paste0("",1:25)) -> webpages
for(x in webpages)
read_html(x) %>% html_nodes("p:nth-child(1)") %>% .[[2]] %>% html_text() -> Company
read_html(x) %>% html_nodes("p:nth-child(2)") %>% .[[1]] %>% html_text() -> Executive
results=rbind(results, data.frame(Company, Executive))
results=data.frame(lapply(results, as.character), stringsAsFactors=FALSE)
results[74,]=c("Trifacta", "Top Executive: CEO Adam Wilson")
results %>% mutate(Name=gsub("Top|\\bExec\\S*|\\bCEO\\S*|President|Founder|and|Co-Founder|\\:", "", Executive)) %>%
mutate(Name=word(str_trim(Name))) -> results
results %>%
select(Name) %>%
findGivenNames() %>%
filter(probability > 0.9 & count > 15) %>% -> data
data %>% group_by(gender) %>% summarize(Total=n()) -> dat
legend="{ position: 'bottom', textStyle: {fontSize: 10}}",
title='TOP 100 BIG DATA COMPANIES 2015
Gender of CEOs',

8 thoughts on “The Gender of Big Data

  1. ‘Like it or not, technology seems to be a guy thing’ feels like a pretty loaded statement. The fact that the companies sampled had a bunch more guys working for them doesn’t say anything about whether big data is something that one gender enjoys more than another.

    1. I came here to complain about that sentence. It felt offensive to me as a woman who works in data analysis and programming. Thanks for beating me to it, James!

      It’s fine if Carmen wants to be a dancer (I hope you praise her for her hard work as much as her beauty), but I don’t want her or anyone else to feel like she’s not supposed to want to be a programmer, president, or anything else she decides to be in the long run.

      1. We both think exactly the same about what we would like for people. I think the sentence may be misunderstood and I don’t have any problem to remove or change it. So please accept my apologies.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s