Sunday 18 July 2010

FanFiction.Net Member Statistics

The research team is proud to present you first numerics from our user-related queries. This post answers many questions, including the following:

-How many writers are there on FFN?
-How long will you stay on FFN?
-How many stories do they write?
-How many users are deleted from FFN for infringement of ToS?
-How quickly does FFN grow?
-How many readers you should expect for a story?

First, we must present the methodology, though. The study consisted of generating 1100 random user account IDs spanning from 1 to 2,400,000 (source data at the bottom). It allows us to generate representative unbiased results at a 95.34% confidence level and a 3% error margin. The list has been generated on the 29th of June 2010. Therefore, we have included all accounts that have been registered, enabled and fully functional, without restrictions of story creation or profile/review posting.

Now, the definitions. You will see the following criteria used in this post:

Empty account: any account that does not host stories uploaded by the owner. In layman terms, there are no stories posted in this account. There may be favourites. Here and here are examples of accounts dubbed 'empty'. Conversely, this is not an empty account.

Active/alive account: any account that has shown signs of life in the past six months, from January 1, 2010. This may be the following: updating or posting a story OR updating the profile OR adding a favourite story OR reviewing a favourite story in the past six months. For example, these two accounts are called 'active' or 'alive' in this post. In the case of the second example, please check the favourites. As long as at least one criterion is met, it is active. Those, who have joined fan fiction in the year 2010 are active by default due to a professional grace period to create a story.

Inactive/dead account: any account that does not meet the active/alive criterion above. Here are two examples.

Deleted account: any user ID that shows the following or similar message "User does not exist or is no longer an active member."

Main Part

You probably recall that FFN has ~3,300,000 stories from our last research (number rounded up to accommodate growth since the previous post), which is 53% of all posted material, with the other 47% deleted. Keep this in mind for a moment.

In the sample of 1100, we have discovered 742 empty accounts, which means, via representativity, only 32.5% of all FanFiction.Net users have stories posted. How does that transfer into general numbers? In a population of 2,400,000 members 781,000 have stories (4.2 stories per account with a story on or 1.375 stories per every member), while the remaining 1,619,000 do not participate in adding content. Two thirds of all members are pure readers, or so it may seem. If it were correct, we could say that 1 writer has 3 dedicated readers on average, if we assume writers themselves read. However, it's not that simple.

Some accounts are plain dead. How many? In a sample of 1100, 855 accounts were inactive, and showed no signs of life in the year 2010. What does that mean for FFN? 78% of all accounts on FanFiction.Net are dead. Less than a fourth, or 22% is currently at your disposal, or 528,000, which is less than the number of accounts with stories on them.

The fun part begins now. How many writers are active? Who could you expect updates from? We connect the overlapping clauses of 'active' and 'not empty'. In a sample of 1100, 130 accounts showed signs of life and had stories on. It translates into: 12% of all accounts on FFN have at least one published story and are actively engaged in fandom activity. 88% of members on FFN are currently not shaping any fandom. As for those, who do, there are 283,000 of them. We have found out that there are 5259 fandoms on FFN, which would mean 54 people keep a fandom alive in the course of 6 months.

On average, no more than 54 people appear in a fandom over six months. How many new people is that per day? 0.3 of a person drops into an average fandom. An average fandom has 681 stories. A median fandom, the one in the middle, which ditches the enormous influence of HP with 0.5 million stories, has 16. That was a bit of extra information, and we now return to users.

One aspect of FFN particularly interested the research team, the number of account deletions by the administrator. 0.73% was the number we acquired. That's less than 1 in 100. However, let us convert that into raw numbers. 17,500. We add an arbitrary 3000 to that number because accounts from 1 to 3000 are unavailable, and the account number generator did not account for it. What do we get? Since September 1998 fanfiction deleted over 20,500 users for infringement. It stands for 0.85% of all users. 4.75 accounts are deleted per day on average, a very modest number because we disregard deletions impossible to document and test easily, like those attributed to policy changes (for instance, when MSTs were deemed unwelcome).

Who would that be? Blacklisted people: spammers, trolls, plagiarists, other infringers. They missed a few trying to use FFN as an advertising venue here and here.

By now, you already know how many account totals are there. It's time to break them into a time series and give you an understanding of how quickly FFN grows.

A table below tackles this issue. We need to explain the columns for complete clarity:

Total: the last account ID created in the year (AKA summary number of accounts created until December 31, all years including the one in the row [accounts made this year + all accounts made in the previous years])
Change: number of accounts that were created in the year in question
Growth%: how much accounts FFN gained in comparison to the previous year, excluding accounts created in the previous years.
CChange%: chained value of change. The ratio of Change (this year to last) divided by the ratio of Total. Answers how quicker (above 1)/slower(below 1) grew this year in comparison with the previous, acceleration.
Middle: the date when half of the annual growth is reached, 50% of accounts created in that year are already present by this date.

Year - Total - Change - Growth% - CChange - Middle
1999* - 6749 - ... - ... - ... - ...
2000 - 33,090 - 26,620 - 411.4 - ...
2001 - 147,200 - 114,110 - 344.8 - 0.19
2002 - 318,900 - 171,700 - 116.6 - 0.16
2003 - 512,000 - 193100 - 60.6 - 0.32 - June 22
2004 - 733000 - 221000 - 43.2 - 0.5 - June 13
2005 - 959000 - 226000 - 30.8 - 0.55 - June 29
2006 - 1188200 - 229200 - 23.9 - 0.63 - June 21
2007 - 1458900 - 270700 - 22.8 - 0.78 - June 17
2008 - 1788000 - 329100 - 22.6 - 0.81 - June 3
2009 - 2238000 - 450000 - 25.2 - 0.89 - May 31
2010** - 2680000 - 442000 - 19.8 - 0.66 - July 21

*Accounts created in 1998 added. It is impossible to tell when exactly a person joined before 2000-01-07.
**estimated, based on the first 6 months.

Before we begin analysing the data, there is an explanation for our 2010 estimate. We calculated it according to seasons, not a plain average. Based on our calculations, by June 21 the site receives 50% of its annual account growth spurt. This means that slightly more accounts are created in the first half of the year, than in the next six months. Site-wide, there is no reason to assume 'big' events like the release of a movie or a new popular book create significant fluctuations. Years before 2002 were not included due to volatility while the site was still young.

Now, let's carry on with the examination. As you can see in the Total column, the site is growing every year. Rational. The Change column shows that an increasing number of people joins the site up to 2010, with the period from 2004 till 2006 being stable in terms of Change. Things become trickier with Growth% and CChange. Some of you may be confused why a site which is growing more and more in raw numbers seems to score poorly in the last two columns. The explanation is as follows: as the site grows, it needs a larger number of new accounts to sustain itself. Simple example: site with 1000 accounts made in the previous year gets 1000 more this year. Next year, it will be 2000 accounts. If the site grows another 1000 next year, this 1000 will be relatively smaller (50% vs 100%) than the first. The same is happening to FFN, as it gains a similar number of accounts that weigh less and less.

The rate of acceleration or slowing down is most visible in CChange. Not a single value is higher than 1, which means the site never grew faster than the year before. On the contrary, the rate of slowing down, the closer to zero the less momentum the site gains compared to last year. From 2000 till 2009, deceleration (slowing down) was becoming closer to 1, a sustainable equilibrium point, but the year 2010 returns us to levels of 2006.

In layman terms, imagine two speeding cars. One of them is the site, and the other is 1, how the site did last year. The other car is a ghost/time challenge type that repeats the race as it was before. The ghost reaches the finish line first every time because your car never reaches the value of 1. You lose one race. Next time, the ghost repeats how you raced the time you lost. And again. Meaning, every race the ghost is slower, repeating your losses. You keep losing, though. While you do, you notice that if at first you lost by a long shot, after several runs, you still lose, but 1 is a lot closer.

If it weren't for 2010, a great gap in a seemingly fluent continuity, we could have made an obvious conclusion that FFN will, eventually, grow faster, and its growth will be bigger both in volume and ratio that volume takes in the whole (your car will start a winning streak).

Regression analysis showed that there is a polynomial relationship between time and growth. Linearly, there is a positive relationship and a linear trendline would claim that the site will reach CChange=1 in 2012. With an R^2=0.825.

A polynomial trend fits better, with R^2=0.9 for the parabola. It means that the function you will see below 'catches' 90% of all vibrations that our growth spurt (CChange) makes, and best describes fluctuations in growth on FFN. What does that R^2 mean? 90% of all growth fluctuations are explained by time in the function below.

y = -0,0094x^2 + 0,218x - 0,4813

y - CChange value

x - number of years since 1998 (0, 1, 2, et cetera)

Basically, this function allows us to calculate the future of FFN. What is it? Well, according to this, the CChange value will be 0 when the site reaches 21 years of age or by year 2019. This is the scenario we follow if the site does not gain momentum by 2012. If we employed descriptive statistics, any CChange above 0.779 and under 0.3 would have been considered anomalous (the rule of three standard errors). Removing those values gives us a more pessimistic, yet less accurate, picture of these events. Reaching 1 would take three years longer linearly, and negative CChange would also be acquired sooner in more reliable polynomial models. Our choice on extrapolation is based on the principle of numeric accuracy, provided other factors remain static. Surely, clever website management and an increased interest in fan fiction as a concept is bound to change the end result. It does, however, suggest that site administration would avoid the trend described in this exercise.

As a final part of this piece of research, we would like to address a number we have shown you before 12%, the number of accounts that have stories on and currently participate in fandom. Another 10% are active readers and do not have any stories posted. This is a general number, though, and we are sure You are more curious to know where do you stand with your peers rather than the whole site.

Below is a table with the following columns:
Year: year of joining.
Full: possibility% that your account is still active and has stories if you joined in the designated year
Empty: possibility% that your account is still active, but has no stories, if you joined in the designated year
Full stays: the probability% that if you have stayed until July 2010, you have stories on

We start from the year 2002, when initial FFN volatility abated. Empty in 2010 is skipped.

Year - Full - Empty - Full stays
2002 - 6.4 - 2.5 - 71.4
2003 - 8.5 - 1.1 - 88.9
2004 - 3.7 - 1.9 - 66.7
2005 - 5.7 - 2.3 - 71.4
2006 - 9.1 - 2.0 - 81.8
2007 - 9.1 - 5.8 - 61.1
2008 - 16.2 - 2.8 - 85.2
2009 - 18.6 - 21.3 - 46.6
2010 - 28.4

Interestingly, you are more likely to stay over a year on FFN if you have stories and are a writer than if you were just a reader. However, you have an equal chance of staying on FFN for a year, writer or reader alike. Regardless, if you join FFN, chances are you will not write a story and you will not be on the site longer than six months.

Even if you have written a story, it is most probable that you will not be on the site longer than six months. This is a generous time period, and it could be that six months is the most probable activity lifespan because it is the starting point and anything smaller does not exist in this part.

We have worked on regression to give you an easy way to calculate the perspectives of staying on FFN. A fifth degree polynomial function seemed to have the biggest R^2=0.99. Amusingly, the probability would go down to negative 1700% very quickly after 8 years, so we had to switch to a simpler parabolic function with R^2=0.96.

Y=0,0218x2 - 0,2603x + 0,961

x - the number of years you have/are intending to stay on FFN. (Works for values up to 10 years).

y - % that you will stay.

According to the given function, it is least likely that you will stay on FFN for 6 years. Thus, yes, more likely that it will be 7 or 8. We attribute this to some form of fandom patriotism the earliest members have expressed to the site. A more precise function would have to include account deletions, which should, in reality, lower active account rates (remember the 3000 first accounts?) and the possibility of staying much longer than 8 years. In any case, the function above is presented for your amusement. A more informative variant is below.

We understand that it might be difficult to imagine the contextual difference between 6% and 9% dominant in the previous table. For this reason, we have made a coefficient, so 28.4%=1. This way, you will see more clearly how many active accounts die away, and how many stay active.

8 years 23%
7 years 30%
6 years 13%
5 years 20%
4 years 32%
3 years 32%
2 years 57%
1 year 65%
0 years 100%

The process can be done further if you want to see how many % of 65% et cetera die in the following years.

Active fanfic participating accounts (those that make up 12% on the site, remember that) lose 35% of their numbers in the first year. The second most rapid drop is in 3 years, but people who tend to stay 3 years are prone to staying 4. The last accurate piece of data that coerces with the trend: the more time passes, the less people stay, is 6 years. Only 1/8 of the people who are active writers right after joining remain this way. 7/8 chip off during the trip. As such, the number of permanent contributors (who stay on the site for years) increases as FFN grows. There is only one 'but': the increase is majorly consumed by users abandoning their accounts.

Those, who have spent less than 6 months account for 6.5% (29.5%) of the 22% of people that are active in any way. Another 7.3% (33.1%) come from those, who have spent more than a year. As such, it is reasonable to say that almost two thirds of the site is actively inhabited by inexperienced account owners, rated 'fans' in forums. So-called 'fanatics' make up a third of the active population, a third that spans since 1998 till the beginning of 2009. On the one hand, it is peculiar that the amount of active newbies (writers or just readers) is almost equal to that of 'fanatics'. On the other, it should make quality control out of the question. Why does it not even out? A question we leave in your hands, dear readers.

Conclusion

Unless FFN manages to speed up its growth potential, those 12% that currently shape the fandom will not be enough, especially because ~5 accounts are deleted every day. The site needs to replace more than 35% of active users every year, and 2010 so far looks the most challenging yet. More dedication, fellow fans. May the concept of fan fiction prosper.

Added: here is a list of user accounts in our sample.

Question: What about people who just go to forums, aren't they active?
Answer: They do not make use of the site's core service as a fan fiction archive. If you don't write or read stories, you are considered inactive. The only way a forum goer could be included as active (provided they have no stories or favourites) is if they updated their profile this year.

Wednesday 7 July 2010

Most Popular Categories

This post is to clear confusion on FFN about what is popular on FanFiction.Net, what is not, and why. All statements you will see in our report are based on raw point data, collected on 29th June, 2010. This means that everything has been taken in machine fashion directly from the site in the method known as observation, not via sample by omitting fandoms. (It was scooped by looking at the numbers, for the younger readers.)

As such, the presented data has a 100% confidence level. However, we understand the value of server delays and are including an arbitrary 3% error margin because the data is taken from the top-category view, not by trawling inside every fandom. If you recall from the previous post, top-category views show a slightly bigger number of stories (up to 5% for some fandoms) for active fandoms (with over 50 stories), so this is included due to FFN site dynamics as a precaution, despite the inclusion making no difference statistically. This is made necessary further because a part of our target audience has not passed a statistics course.

To make this more interesting, we suggest that everyone takes a guess, which top category is more popular: Anime/Manga, Books, Games, TV shows et cetera, from the list of 10 on the front page.

Depending on your age, the answer is probably 'Games' or 'Books'. The answer would not be far from truth, but not even Harry Potter, the biggest fandom on the site gives Books the top spot. Conversely, it's not the combination of Kingdom Hearts, Pokemon and other games, all with the biggest forums on FFN.

The largest top-level category on FanFiction.Net is Anime/Manga with over 1,062,835 publicly available stories. It also has numerous subcategories/fandoms, 955. Despite this, a third of them (over 300) has under 10 stories, with the bulk concentrated in Naruto 240,635 and Inuyasha 93,196.

If you recall, FFN has approximately 3,200,000 live stories, which means every third story found on FFN is related to Anime/Manga. Why? We can't answer that just yet like we can't tell you why only 1 in 50 writers becomes a Beta Reader.

Anime and Manga have #1. Now, for all Harry Potter, Twilight, LotR, Warriors, PJO et cetera fandoms, you are not unimportant. Books have a firm #2 with 811,044 live stories. Respectively, 461,311 and 150,708 belong to Harry Potter and Twilight. Let's dwell on these two for a moment. The HP fandom is marginally three times as big as Twilight. The underdog may claim this exists because Harry Potter has been a lot longer than Twilight. Making matters fair, that would mean Twilight would be as popular as Harry Potter if they were of the same age.

What's real and what's not? The first HP book has been released in 1997, 13 years from now. The first Twilight book has been released in 2005, 5 years from now. One might exclaim: "Ah hah! Two years is a very small time, so they must be equally popular!" Let's do the math. HP is 2.6 times older than Twilight. Had they been released at the same time, Twilight would now have 391,000 stories, 70,000 behind HP. How big/small of a difference is that? That's almost two LotR fandoms and the total number of new books released in Spain annually.

We return to weights. If the largest fandom in Anime/Manga, Naruto has 240,000 stories, a fourth of the total Anime/Manga, HP has 461,000, way over half of all Book-related fiction accessible on FFN. One may think, seeing that books are #2, more popular than Games, Comics, TV shows (some of which summed up), the world of fiction is into literature fandoms. False. Anime/Manga is popular because there are many fandoms. Books is big because there are many HP. In layman terms, it would be sensible to rename 'Books' into Harry Potter & Twilight, pop reads, which can only scratch the surface of, say, critically acclaimed classical literature. The audience on FFN could have been assumed as an active participant in literature fandoms. It is, however, an active participant in HP and Twilight-level literature fandoms.

Moving on to #3, which is TV shows at 580,596. Curiously, their outlook is similar to that of Anime, with a third of all fandoms having under 10 stories, and there being multiple weight leaders. 15 fandoms take the range from 40k to 10k. In Anime, 20 fandoms have over 10k stories. In Books, 4 fandoms have more than 10,000 stories.

One may want to run an economic monopoly concentration index formula on these numbers. In case it is viable, we present the number of fandoms in the top categories.

Anime/Manga: 1,062,835 stories; 955 fandoms; 20 fandoms have above 10,000 stories
Books: 811,044 stories; 1138 fandoms; 4 fandoms have above 10,000 stories
TV: 580,596 stories; 1013 fandoms; 15 fandoms have above 10,000 stories

Additional research is suggested for the curious: remove all the popular fandoms that add substantial weight to the category (have above 10,000 stories) and make an account of how much dead weight, or impopular fandoms, every top level category has.

At the moment, numbers suggest that the Anime/Manga is the healthiest fandom. Why? 1. It has more stories than others. 2. It has less fandoms. 3. It has more popular fandoms than Books and TV altogether. 4. It is least threatened by C&D (cease and desist) letters.

The fourth one is important. If a cease and desist letter is sent to, say Harry Potter fans, forbidding them to write fan fiction, books would drop dramatically from 811,044 to 349,733. If the same happened to an Anime/Manga fandom, the most loss it would have would be 240,635, less than a quarter of its size. Same applies to TV.

Let's not ignore other fandoms. Below is a limited table/list of top categories without crossovers. Crossovers were counted in our summary of all stories on FFN, but they produce confusing data in the way they are organised, belonging to several fandoms at the same time. The list below is made for clarity purposes.

Name - Story number - Fandom number - Fandoms above 10k - Top fandom

Anime: 1,062,835 - 955 - 20 - Naruto
Books: 811,044 - 1138 - 4 - Harry Potter
TV: 580,596 - 1013 - 15 - Buffy: The Vampire Slayer
Games: 269,261 - 614 - 6 - Kingdom Hearts
Cartoons: 192,918 - 320 - 5 - Teen Titans
Movies: 125,000 - 943 - 4 - Star Wars
Misc: 105,500 - 34 - 2 - Wrestling
Comics: 33,824 - 123 - 0 (8 above 1000) - X-Men
Plays: 15,300 - 85 - 0 (5 above 1000) - RENT

The following information is suggested for further study: how many fandoms are uninhabited (have below 10 stories), how much is that divided by the number of fandoms in the category?

An explanation should follow for the last two categories, Comics and Plays. Plays, for example, have been included much later than other top categories, setting them aside. Also, since they do not exceed 100,000 stories and it is rational to use a proportion size, the active fandom definition is adapted to them as 1000.

As always, present your questions, solicit ideas. This blog is interactive, and we will cover your topics of interest. Coming up next: how many writers does FFN really have?

Tuesday 6 July 2010

FanFiction.Net story totals

Good news!

Our research venture has completed gathering data about site-wide story numbers. This post explains how many stories FanFiction.Net (FFN) really has.

The data in our evaluations has been generated based on the total number of stories posted on June 25th, 2010. The gathered data has been in processing since June 25th, 2010 till June 30th, 2010. We treat it as spatial or point collection; 5 days = 1 instance.

At the time of collection, there has been a total of 6,085,534 registered story entries, based on the newest registered story number in the Just In section on June 25th, 2010.[1].

However, we understand that some stories are deleted, and their ID number is not taken out from the database to be recycled for a new story.[2] Instead, the list carries on, and every newly posted story receives a number higher than the previous.

(An additional explanation for younger readers: you submit a story, and it gets an ID number in FFN's database, so everyone could easily find them. Let's say your ID is 123. If you know that, you can easily make a link without having to copy anything because all stories on FFN have http://www.fanfiction.net/s/*your story ID*. When someone posts a story right after you, their ID is 124, then 125, 126 and so on and so forth. Say, the site got to story number 140, but story 128 has been deemed illegal because it was about living actors, and deleted by the FFN staff, so nobody would sue. What do we have? We have numbers from 1 till 140, but 128 has been deleted. You can't know it has been deleted, by the way, because you're not the writer, and the only way to find out is to check. There are now 139 stories on the site even though it looks like there is 140. Thing is, on a site as big as FFN, you can't just guess how many numbers are 'blank' like that.)

It is the main reason for this analysis: the number FanFiction.Net presents to you is not the total number of stories it has at the moment, but a sum of all fanworks it had at every moment of time available to the public. The key term is 'available to the public' because FFN, according to their ToS, keeps server copies of user submissions. It is reasonable to assume that the real number of stories we can see now (dated June 25th, 2010) is not over 6 million.

We're implementing two methods to reach the data. The first is doing an account of all stories present in all ten top categories and crossovers such as this. Surely, it is a lot of very repetitive and dull labour, but it gives us the exact number, which is: 3,256,278 stories.

As of June 25th, 2010 there are 3,256,278 stories noted as accessible to the public on FanFiction.Net.

This is an accurate number, but it is not 100% of what the story number has been. Why? We made a top category account, without having to rummage through every single fandom, opening it like this. Why is this important? The number of stories in the top category window is always bigger (or even, when the fandom is inactive [has less than 50 stories]) than the real number of fictions one can browse inside the category. The researchers cannot provide you a firm answer on this discrepancy, but it may be attributed to dynamics of stories being deleted at a slower rate than they are added (for example, if you upload a story by mistake, and delete it, you raise the top category number of stories, and it stays above the real number even though you can no longer find the story, a server delay).

It has been determined that, depending on the fandom, the real number is from 0,19% to 5% smaller than the one provided. In large categories, the weight of which forces the researchers to consider them, this number teeters closer to the first value. Now, it might not seem substantial, but Twilight with its 150,000 uploads may have up to 2000 dead stories counted as alive every day. To be completely fair to the estimate, we are multiplying the number by an arbitrary 0.987 coefficient, which best describes the current number of stories, as seen in ten most popular, story-wise, subcategories of Books, Anime and others, except crossovers. Since they make up the trending bulk of FFN, their averages have been considered.

Here is a better estimate, statistically not different than the first, but more exact for the human eye: 3,213,946.

What does that say to you? FanFiction.Net is only 54%-53% (without/with 0.987 coefficient) of what it appears to the layman, with the remaining 46%-47% being deleted content. As such, you may take it that every second story is destined to be deleted, and out of every two stories You post only one will survive (statistically).

What about the second method? Aside from these real numbers taken in raw, the research includes a sample of 1100 randomly generated story IDs with a range [1;6085534], which allows the research to continue with case study at a 3% error margin and a 95,34% confidence level. The survivability estimate taken from the sample size is 55%, which is within the 3% acceptable error and statistically identical to 54%-53%, received with the help of raw data. For future studies, this means our method of sampling follows the general population's characteristics.

In conclusion, there are 3,213,946 stories on FanFiction.Net at the time of our study, and nearly half of all stories posted will sooner or later disappear. How soon? Come back later to find out!

Should you require additional data, requests can be made in the comments, emailed to Lord Kelvin or posted in the Literate Union forum. The list used in our sample can be found here:
http://www.usbupload.com/23228_FFNstatsdatadoc.usb
http://www.usbupload.com/23227_samplelinksFFNdoc.usb