Discovering the Temporal Patterns in the Use of Flickr in Amsterdam
SUBMITTED IN PARTIAL FULLFILLMENT FOR THE DEGREE OF MASTER OF SCIENCE
H
SU
Y
OUNG
H
O
10867619
M
ASTER
I
NFORMATION
S
TUDIES
H
UMAN-
C
ENTERED
M
ULTIMEDIA
F
ACULTY OF
S
CIENCE
U
NIVERSITY OF
A
MSTERDAM
August 18, 2015
Supervisor
dr. Stevan Rudinac
Intelligent Systems Lab Amsterdam,
University of Amsterdam
Second examiner
dr. Peter Weijland
Informatics Institute,
Discovering the Temporal Patterns in the Use of Flickr in
Amsterdam
Hsu-Young Ho
Dept. Information Studies University of Amsterdam Amsterdam, The Netherlands
hsu-young.ho@student.uva.nl
ABSTRACT
The amount of user-contributed photos has increased signif-icantly in social media websites due to the proliferation of smart phones. Mining this quantities of diverse social me-dia with the rich metadata is a good way to understand the user behaviours and overview life in a city. The goal of our case study is to discover the temporal patterns in the use of Flickr in Amsterdam over 11 years and identify the major events based on tag occurrence patterns. This research is mainly divided into two parts: in the first part, we visualise the use of our dataset by different temporal granularity and further investigate the user-contributed tags in some inter-esting time periods. In the second part, we select represen-tative event-related tags and then analyse their occurrences. Our experiments show several informative visualisations so as to provide the insights in our dataset. Finally, we gather information from other sources, namely weather, national holidays, and events, and investigate existence of correla-tions with our dataset.
Keywords
Temporal information, temporal patterns, tag co-occurrence, social media, visualisation, structure mining
1.
INTRODUCTION
The amount of user-contributed photos has increased signifi-cantly in social media websites such as Facebook, Instagram, and Flickr as a result of the proliferation of smart phones. According to Kleiner Perkins Caufield & Byers (KPCB) an-nual trends report for 2014, 1.8 billion photos are uploaded and shared every day by web and app users [22]. Flickr1, for example, which is a representative social photo sharing site, as of 2014 has 92 million users and around 1 million photos were shared every day [10]. In addition to hosting the photos, Flickr also allows users to annotate their pho-tos with textual information such as title, description, and tags. Most of the photos contain rich metadata such as size, timestamp when photo was taken, and camera type. Furthermore, the geotags, which are recorded in the form of latitude and longitude coordinates, are becoming stan-dard metadata owing to the new digital cameras and smart phones with built-in GPS.
Mining these great quantities of diverse social media and the rich metadata offers a possibility for understanding user behaviour and overviewing life in a city. In this research,
1
http://www.flickr.com
using Amsterdam as a case study, our goal is to analyse the quantitative use of Flickr over period of 11 years and identify the major events based on tag occurrence patterns. The main research question is:
What are the temporal patterns in the use of Flickr in Amsterdam?
In order to answer the research question, we investigate the factors like season, day of the week, time of the day, daily temperature, weather conditions and national holidays that we expect the data will show the correlations with. We take the metadata about image capture date, which is provided by a camera, so as to detect the factual time of user activity. Though the camera settings may be incorrect, here we do not take this into account.
Besides the quantitative analysis for investigating the above-mentioned factors, we are interested in finding events in Am-sterdam, triggering the bursts in the use of Flickr. Flickr proves that users are willing to tag their photos to provide the semantic context, for example, the descriptions of ac-tions and events, identities of objects, people, and groups, and sentiment, in order to make their posts/ photos better exposed to the general public [26][2]. Thus, tag plays an im-portant role in retrieving data from social media. However, tag is freely defined by the user and there are many photos on the same subject with a wide variety of tags in Flickr. We fo-cus on finding event-related tags in interesting time periods of different granularity, such as day, week, month, and year, and further discover their tag co-occurrence patterns. For example, the activity-related tag such as “queensday”, which refers to a national holiday in The Netherlands, will occur frequently in the end of April every year. Moreover, the re-lated tags such as “koninginnedag”, “orange”, “holland”, and “party”, will co-occur in the similar period.
In this research, we also visualise the temporal patterns us-ing different techniques in order to obtain the insights into the data. According to Few [11], a successful data visualisa-tion can be easily seen by our eyes and easily understood by our brains. We followed the 9 essential design rules in statis-tical graphics, proposed in [28], for our visualisations. Our aim is to reveal the temporal patterns in the use of Flickr in Amsterdam, and thus predict the possible patterns in the future. Finally, we collect information from other sources in attempt to find the causes behind the user behaviours and temporal patterns. These sources include natural
phenom-ena, such as the temperature, sunshine duration, rainfall, and day length, as well as the Dutch national holidays and annual events in Amsterdam.
The rest of this research is organised as follows. We review related work in Section 2. We describe the methodology in Section 3. Experiments and visualisations are shown in Section 4. Finally, we discuss the result in Section 5.
2.
RELATED WORK
The field of studying social dynamics of a city using social media data had evolved in the past decade. Several studies used spatio-temporal metadata to discover a city. Ratten-bury et al. [25] made an attempt to extract both place and event semantics from Flickr spatio-temporal metadata. Cranshaw et al. [6] studied the dynamic nature of human activity patterns in Pittsburgh, Pennsylvania by exploiting millions of Foursquare check-ins. Li et al. explored the demographic and socioeconomic patterns in the behaviour of the local residents in California by using georeferenced tweets and photos collected from Twitter and Flickr [19]. In [16], the authors illustrated different aspects of NYC with regard to trends, events, food, wearing and transportation with Twitter, Instagram, TripAdvisor, and open data. How-ever, since our focus in this research was finding temporal patterns in the use of Flickr and detecting major events in Amsterdam, we here concentrated on extracting temporal metadata. Discovering temporal patterns and the regularity of the dataset required segregating timestamps with multi-ple granularities by using their repetitive peculiarity, such as the hour, the week, and the month [20][17]. Silm et al. employed mobile-phone positioning data of 5200 residents of Tallinn over three years in order to find the temporal varia-tion of ethnic segregavaria-tion across different granularities [27]. By aggregating data over time, Wang et al. built an interac-tive visualisation to present temporal summaries in multiple granularities [30].
Lately, social media sites established the services which allow users to annotate web resources, such as Del.icio.us2for
web-page bookmarking and hashtags used in Flickr and Insta-gram for photos sharing [33]. Investigating user-contributed tags then became one of the popular topics when study-ing the dynamics, structure, and semantic context of social media sites. Firan et al. made use of tags for classifying Flickr photos into different event categories [12]. One of the projects of Yahoo! Research3 developed an interactive
real-time web-based application to visualise the most repre-sentative tags in different temporal granularities [8]. User-contributed tags were noisy, which was a challenge addressed in [5][15][2], for the reason that they were freely defined by the users. Additionally, users can annotate a photo with several tags, while different tags may be annotated a same subject by different users. Many existing works used these tag co-occurrences to discover the relatedness between the images [31][3][34][32][4]. Another approach in [14] had ap-plied tf-idf weighting to reduce the noise so as to find the representative tags of a cluster that contained Flickr photos captured in a given geographical area of San Francisco.
2http://delicious.com 3
http://research.yahoo.com/taglines
Our research is related in spirit to that of [23], which anal-ysed 4.25 billion of Flickr photos by different temporal gran-ularities so as to find the temporal patterns. Similar to the approach in [23], we also analysed the number of Flickr pho-tos captured in different time periods, but our research is complementary in that we also focus on the semantic con-text by extracting the social tags and finding correlation between different sources. Eisinga et al. analysed the data of voter turnout in all national parliament elections held in the Netherlands based on temperature, sunshine duration and rainfall on election days [9]. The analysis showed that the weather parameters affected the willingness of voters. We then assume that weather conditions could influence the activity of Flickr’s users too.
3.
METHODOLOGY
Our approach to finding temporal patterns in the use of Flickr in Amsterdam consists of the following six steps. 1) Preprocessing. We preprocess dataset by selecting related parameters and filtering the data with the same time period and location. 2) Quantitative usage detection. We group the data by different temporal granularities so as to find the pat-terns in the temporal distributions. 3) Bursty tags detection. We analyse the tags in some interesting time periods iden-tified in 2). 4) Event analysis. We choose some known tags and investigate their occurrences. 5) Visualising. By sup-porting visual representations of our data, we can provide interesting insights. 6) Finding correlation. We compute Pearson correlation for finding the correlation between our Flickr dataset and other sources.
3.1
Data Description and Preprocessing
In this research, we made use of a dataset from [29] with 160,352 Flickr photos associated with their metadata cap-tured within Amsterdam. We filtered the data from January 2004, which was the founding year of Flickr, to December 2014 in order to discover the temporal patterns over 11-year period. In total, there were 128,841 photos with their asso-ciated “photo id”, “description”, “date taken”, “title”, “url”, “longitude”, and “latitude” as well as 898,377 tags. Next, we converted the iso formated dates into Python’s datetime type in order to allow further analysis.
Besides the main Flickr dataset we described above, we col-lected information from other sources, which were possibly relevant to our temporal patterns. First, we collected the weather information of Amsterdam Schiphol meteorologi-cal station, which is located at latitude: 52.301N, longi-tude: 4.774E, from Royal Netherlands Meteorological Insti-tute (KNMI) between 2004 and 2014 [13]. We selected a list of climate variables, namely ”daily mean temperature”, ”min-imum temperature”, ”max”min-imum temperature”, ”percentage of maximum potential sunshine duration”, ”daily precipitation amount”, that we were interested in as our weather’s dataset. Second, for the purpose of comparing the day length in dif-ferent seasons with the use of Flickr in Amsterdam, we gath-ered information from dateandtime.info [7]. The dataset consists of the temporal information of sunrise, sunset, solar noon, day length, and twilight from 2004 until 2014 in Am-sterdam. Finally, we retrieved the information about Dutch public holidays and annual events in Amsterdam from “I
amsterdam”4.
3.2
Trend and Event Detection
We used bottom-up method to detect the trends from Flickr with multiple granularity. The data was grouped by month, weekday, and hour of the day to examine the user’s behaviour and find the peaks in the temporal distributions. Furthermore, we investigated existence of interesting peri-ods in user behavioural patterns. On the other hand, we used top-down method to analyse some known tags. In our research, we analysed two known event-related tags in Amsterdam, which are queensday and gaypride, by investi-gating their temporal distributions and tag co-occurrence. On the one hand, Queen’s Day (Koninginnedag) is one of the most widely celebrated holidays in the Netherlands and attracts more than 700 thousands visitors every year. On the other hand, the Netherlands was the first country in the world to legalise same-sex marriage [24] and Amster-dam Gay Pride is a citywide LBGT festival held annually at the center of Amsterdam. Therefore, we assumed both events are well-known and distinctive in Amsterdam. Before starting our tag analysis, we first visualised all the tags in our dataset as shown in Figure 1a to get an overview of our tag corpus. It is easy to see that the dataset contains many trivial and uninformative tags, such as “holland”, “am-sterdam”, “netherland”, and “nederland”. Figure 1b shows the word cloud which is based on Figure 1a but excludes those non-informative tags. Next, we obtained a list of gen-erally popular tags from Flickr (see Figure 2) and filtered them out (as seen in Figure 1c) in order to reveal the event-related tags for Amsterdam.
Figure 2: A list of generally popular tags according to Flickr
3.3
Tag Analysis
In this research, we focused on finding the informative tags instead of classifying visual content. The main challenge in tag clustering was that tags are freely defined by the user so there were many photos on the same subject with a wide variety of tags in Flickr. Therefore, we applied the following algorithms to reduce the noise in tags, aiming to find the representative events with different temporal granularity.
• TF-IDF Weighting
Term frequency-inverse document frequency (tf-idf) is
4
http://www.iamsterdam.com/en
a statistic measure, which is often used in information retrieval and text mining. Tf-idf is the product of term frequency and inverse document frequency that aims to find the important words or terms in a document based on how frequently they appear across multiple documents [18].
To begin with the tag analysis, we let Gτ be a set of
temporal granularity which included a time period of year, month, day, or hour (see following conditions)
Gτ = {y, m, d, h} 2004 ≤ y ≤ 2014 J anuary ≤ m ≤ December M onday ≤ d ≤ Sunday 0 ≤ h ≤ 23
We choose two different levels of τ from Gτ, denoted
by τ1 and τ2. We first filter a sub-dataset C based
on Gτ1 and then divide C into several clusters based
on Gτ2. For example, we choose “April” (Gτ1 is equal
to April) as our sub-dataset C, which is a corpus that contains all the tags assigned in April; and then we divide C into 11 clusters C∗based on Gτ2, which is the
level of year. Therefore, we can assign tf-idf weights to the tags in a specific cluster, such as year 2006, so as to find the representative tags in that cluster by Equation 1. Since Gτ can be any time period as long
as it is satisfying the conditions, we can also choose a time period of 2004 until 2014 as Gτ1, in which case,
the sub-dataset C is same as the initial dataset. tf (t) = kft∈C∗k k∀t ∈ C∗k idf (t) = log kC ∗k 1 + kC∗⊇ tk (1)
In Equation 1, the term frequency tf (t) computes the number of times a tag appears in a cluster C∗, which is based on a level of granularity Gτ2, normalized by
dividing by the total number of tags in C∗. The inverse document frequency idf (t) is designed to measure the common tags. Besides the stop words (e.g. the, is, at), the tags ”amsterdam”, ”netherlands”, ”northholland”, etc will be regarded as common tags in our case due to the high frequency in every cluster. We take the ratio of the total number of clusters (i.e numbers of C∗) to the number of clusters containing the tag, then take the log of that. Also, we add 1 to the divisor to prevent division by zero.
• Co-occurrence Algorithm
Users generally annotated a photo with several tags. There are 898,377 tags with 128,841 photos in our Flickr dataset (i.e. 7 tags per photo on average). On the other hand, different tags might be assigned to the same event by different users. If one photo was tagged by both t1 and t2, it means there is a co-occurrence
between t1 and t2.
The experiment has been divided into two parts by us-ing bottom-up method and top-down method so as to find different aspects of tag co-occurrence. In the first part, we investigated a sub-dataset in an interesting time interval and created a co-occurrence matrix. Fig-ure 3 shows an approach for generating co-occurrence
(a) Uncleaned word cloud from 2004 until 2014
(b) A word cloud with non-informative tags filtered out
(c) A word cloud with the generally popular tags filtered out
Figure 1: Visualising the tags using word clouds
matrix. Given different photos (P1, P2, ..., Pn)
an-notated by different tags (t1, t2, ...,tj), the pairwise
co-occurrence, co occur(ti, tj), is computed by simply
counting the number of photos that are tagged with both ti and tj. The diagonal shows the number of
photos with a given tag. For each row, the non diag-onal elements show how many photos have both the row tag and the column tag. For example, tag t2 and
tag tj−1 are both associated with P1, P2, and P3 in
Figure 3.
In the second step, we used top-down method to select a known event-related tag, such as “queensday”. We retrieved all the photos, which were annotated by this given tag, as a sub-dataset. Since every photo in this sub-dataset was annotated by the given tag, we simply counted the top-30 frequent tags.
Figure 3: Tag co-occurrence matrix generating
3.4
Correlation
We applied Pearson correlation for finding the relationship between the temporal patterns we got from our Flickr data and three other sources we described in Section 3.1. Pearson correlation is commonly used in statistics to measure how strong is the linear association between two variables. The Pearson correlation coefficient, which is denoted by r, can be calculated using Equation 2. The value of “r” will be
between -1 and 1; a value of 0 indicates that there is no relationship between two variables. A value greater than 0 indicates a positive association, and a value smaller than 0 indicates a negative association.
r = r n(P xy) (P x)(P y)
nP x2 (P x)2nP y2 (P y)2
(2)
We examined four weather parameters for possible existence of a correlation with the number of Flickr photos. The pa-rameters are monthly mean temperature, monthly sunshine duration, monthly mean precipitation amount, and monthly day length, which are denoted by µt, sm, rm, and lm
respec-tively.
4.
EXPERIMENTAL RESULTS
The research, using a Flickr dataset described in Section 3.1, was performed with the use of Python. More specifically, we used matplotlib [1] and Plotly Python Library [21] for visualisations. In this section, we analyse and visualise the data by the methods we describe in Sections 3.2 and 3.3.
4.1
Quantitative Usage in Flickr
• Monthly Patterns
We used bar chart instead of line graph to display the amounts of photos being taken in each month over 11 years since each monthly cluster of photos was inde-pendent and discrete. A bar chart in Figure 4 shows the total amount of Flickr photos being taken from 2004 to 2014 in Amsterdam by each month. By do-ing this, we aim to find the burst in a year. There are peaks in April and August, whereas it is surpris-ing that there is a trough in July. Therefore, we will investigate those interesting months.
Next, we retrieved the tags of “queensday” and “gaypride” from our dataset and visualised their monthly occur-rence. Figure 5 clearly shows that “queensday” and “gaypride” were highly annotated by the users in a specific month, which is April and August respectively. As a result of both events being well-known in Amster-dam, we can conjecture the peaks in April and August from Figure 4 might be caused by them.
Figure 4: Total Flickr photos being taken from 2004 to 2014 in Amsterdam by each month
Figure 5: The monthly trends of tags queensday and gaypride
Line graphs depicted in Figure 6 show the ratio be-tween the amounts of photos captured each month and the total amounts of photos in the corresponding year. The main purpose of this visualisation was to find the trend and the peak points for every year. As we can see in Figure 6, the number of photos captured in ev-ery month is similar, but July is much more quiet in comparison to other months. However, there is an ex-ception in 2005; April 2006 is also interesting due to an unusual burst in 2006.
Figure 6: Ratio of photos captured per year
• Daily Patterns
We used a radar chart to visualise the daily patterns of each year and a bar chart to show daily trends of tags “queensday” and “gaypride”. Figure 7 shows that Flickr users are more active on the weekends than dur-ing the weekdays in Amsterdam. Figure 8 indicates that there are no particular daily patterns for the tag “queensday”. On the other hand, the tag “gaypride”
was occurring burstly on Saturday.
Figure 7: The trend of total Flickr photos being taken by weekdays
Figure 8: The daily trends of tags queensday and gaypride
• Hourly Patterns
The heatmap depicted in Figure 9 shows that a photo was more likely to be uploaded between 1 p.m and 4 p.m. We also visualised the hourly trends of tags
Figure 9: The trend of total Flickr photos being taken by hours
“queensday” and “gaypride” as shown in Figure 10. The peak hours of Amsterdam Gay Pride are between 3 p.m. and 5 p.m., yet the tag “queensday” occurs constantly during the day. In addition, the number of occurrences of tag “queensday” at the peak hour, which is 10 a.m. as shown in the result, is almost a quarter of the number of tag “gaypride” at 3 p.m..
Figure 10: The hourly trends of tags queensday and gaypride
4.2
Tag Relationships
We proceed by finding the semantic tags in some interesting time periods, which were identified in Section 4.1, by using tf-idf weighting and tag co-occurrence algorithm.
• Representative Tags
We first assigned tf-idf weights to the tags, which were grouped into 12 corpora based on monthly granularity, and visualised the result as a series of word clouds in Appendix, Figure A.1. As observed in the visualisa-tions, some tags are frequently found, such as “rugby” and “football”.
Next, the interesting months and weekdays identified in Section 4.1 were investigated as follows: First, the tf-idf weights assigned to the tags in April, July, and August are shown in Table 1. The columns with “(2004-2014)” in the table were based on monthly tag corpus from 2004 until 2014, which aimed to find the represen-tative tags of each month through 11 years. Second, the columns with a year behind the month were based on a specific month through 11 years. For instance, we filtered the tags occurring in April and divided into 11 corpora based on the corresponding year, to be able to look into a specific corpus (i.e year 2006), which was an unusual years according to Figure 6. According to the result from Figure 4, July was much more quiet compared to other months. As the weather conditions between July and August were similar, we took Au-gust into account in our analysis so as to compare the tags with July. As observed from Table 1, there are more events in August as July.
Second, we examined the tags in 2013, which is the year containing the most photos in the dataset with total 24,113 photos, so as to analyse the difference be-tween weekdays and weekends. We first grouped the photos which were taken from Monday to Thursday in the first cluster and we assigned tf-idf weights to
(a) Monday to Thursday (b) Friday
(c) Saturday (d) Sunday Figure 11: Word clouds of different days in 2013 based on tf-idf
four clusters based on days. Figure 11 displays the word clouds of each cluster based on the results of tf-idf (see details in Appendix Table A1). By observing the word clouds and the list in the table, we found that less representative event-related tags were shown in “day 1-4” than the clusters of the weekends. For example, the top tf-idf tags (i.e the most representa-tive tags) in “day 1-4” are “sonyslta77”, “sonya77 ”, and “sonyslta77v”, which are related to a type of cam-era; and “hogekwaliteit” meaning high quality in Dutch. In contrast, several “football” related tags, such as “voetbaltoernooi” (football tournament), “wvhedw” (a football club), “voetbalwedstrijd” (football match), and “fcalmere” (a football cluc) were annotated on Friday and Sunday. It is interesting to find that “gay” related tags were highly tagged on Saturday even though Am-sterdam Gay Pride is held only during few days in August.
• Tag Co-occurrence
We selected the top-30 bursty tags of April 2006 and July 2005 from Table 1 to discover their co-occurring tags. We created their tag co-occurrence matrices by the method we described in Section 3.3. Figure 12 shows that “receptie”, “scapino”, “opening”, “jozefschool”, and “feest” frequently co-occurred in April 2006. We conjecture that those tags were possibly assigned to the same event. A co-occurrence matrix of July 2005’s depicted in Figure 13. It is interesting to see that the tag “friends” co-occurred more frequently with “home”, “party”, “garden”, and “balcony” than the tag “family”.
Secondly, we selected two known event-related tags, which are ”queensday” and ”gaypride”, and attempted to find the top-30 pairwise co-occurring tags in our dataset. The two bar charts in Figure 14 show the
Table 1: Tags ranking according to tf-idf
April (2004-2014) tf-idf April (2006) tf-idf July (2004-2014) tf-idf July (2005) tf-idf August (2004-2014) tf-idf 1 jozefschool 0.02252 scapino 0.2343 lbgt 0.01113 lastfriday 0.11498 canalparade 0.01166 2 receptie 0.02053 receptie 0.2343 agp11 0.01113 infonl 0.0924 footballmatch 0.00909 3 scapino 0.02053 jozefschool 0.17858 waypoint 0.00911 nokia6230 0.08763 soccermatch 0.00909 4 sportdag 0.00858 opening 0.17858 7052010 0.00651 cameraphone 0.06823 huawei 0.00909 5 koninginnedag 0.00564 feest 0.06212 7062010 0.0039 lunch 0.01187 afcnautilus 0.00907 6 ecir2014 0.0042 2006 0.0077 taxonomy 0.00309 jona 0.01125 soccerteamhuaweiamsterdam 0.00905 7 bredewegfestival 0.00367 botanicgarden 0.00713 hetkwartieramsterdam 0.00291 balcony 0.01022 gaypride 0.00665 8 bredeweg 0.00367 botanical 0.00692 parkeeroverlastprolympischstadion 0.00291 home 0.00928 canonef70200mmf28lisusmlens 0.0046 9 scmuiden 0.00317 hortusbotanicus 0.00544 lastfriday 0.00206 party 0.00762 vrijgezellenfeestjasper 0.00439 10 kingswoodrfc 0.00312 botanischetuin 0.00497 delangstevegetarischetafel 0.00168 lat=523713696066174 0.00625 gayprideamsterdam 0.00369 11 queensday 0.00294 plantagemiddenlaan 0.00497 11juli2012 0.00168 plazeinfonl 0.00625 pride 0.00314 12 sonya77 0.00288 nikonem 0.00462 themakingoff 0.00162 sintantoniesbreestraat 0.00625 canalparade2012 0.00292 13 sonyslta77 0.00288 nikonem 0.00462 ldiaonbass 0.00162 long=490051 0.00625 voetbalwedstrijd 0.00274 14 sonyslta77v 0.00288 nizozemsko 0.00412 nokia6230 0.00154 maarten 0.005 fcalmerea1 0.00274 15 koningsdag 0.00277 botanic 0.0041 lat=5231390974 0.0015 plazes 0.00476 deplaat 0.00227 16 weekendvandesterren 0.00275 isookschitterend 0.00389 clubziggo 0.0015 summer 0.00397 fab6boatcompetition 0.00212 17 olympische 0.00258 unfound 0.00389 ziggolive 0.0015 grapevine 0.00375 gayparade 0.00209 18 linnaeusparkweg 0.00227 portraits 0.00389 lon=493910551 0.0015 work 0.00374 gaypride2013 0.0019 19 april2009 0.00227 gardens 0.00379 theheinekenexperience 0.0015 garden 0.00368 uitmarkt 0.00189 20 opening 0.00209 nederlandene 0.00379 vrijgezellenfeestjasper 0.00132 friends 0.00338 homoemancipatie 0.00184 21 england 0.00197 glasshouse 0.00367 waypointcentraalst 0.0013 puma 0.00286 amsterdam2012 0.00177 22 toulon 0.00195 galenstraat 0.00346 waypointdamsquare 0.0013 luchtfoto 0.0025 bouvy 0.00177 23 muguet 0.00195 moord 0.00346 waypointredlight 0.0013 vrolikstraat 0.0025 uitdekast 0.00177 24 clubdumuguet 0.00195 botany 0.00324 europe2006 0.0013 pancake 0.0025 vacaciones 0.00176 25 april 0.00166 invernadero 0.00324 waypointsexmuseum 0.0013 roofgutter 0.0025 outofthecloset 0.00175 26 hogeweg 0.00139 serre 0.00324 waypointnearafe 0.0013 inktslaaf 0.0025 botenoptocht 0.00173 27 schilderlust 0.00129 hothouse 0.0028 nolimit 0.00128 200507 0.0025 gayliberation 0.00142 28 martineau 0.00129 greenhouse 0.0028 nikkor24120f4vr 0.00127 family 0.00222 agost 0.00127 29 wwwantonmartineaunl 0.00129 springtime 0.00218 videoshoot 0.00125 boxhead 0.00191 outdoors 0.00125 30 games 0.00123 botanicalgarden 0.00216 behindthescenes 0.00125 friend 0.00161 hartjesdagen 0.00125
Figure 12: Co-occurrence matrix of April 2006
top-30 co-occurring tags with ”queensday” (Figure 14a) and ”gaypride” (Figure 14b) respectively. The results show that most co-occurring tags are reasonable re-lated to the given tags in both cases.
4.3
Correlation
We preprocessed the data before computing Pearson correla-tion. First, we counted the amount of photos by each month. Next, we computed the ratio, denoted by Am, between the
amount of photos taken each month and the total amount of photos in the corresponding year by Equation 3. We had several weather parameters, namely daily mean temperature in degrees Celsius, sunshine duration in percentage, daily precipitation amount in millimeters, and the day length in hours, which were gathered from the corresponding sources as described in Section 3.1. We recalculated those daily data
Figure 13: Co-occurrent matrix of July 2005
into average monthly data. The sample size of each inde-pendent variable, denoted by µt, sm, rm, and lm, was 132.
(i.e. 12 months multiplied 11 years) mi
P12 i=1mi
∀mi∈ Y earj, 2004 ≤ j ≤ 2014 (3)
Table 2 shows the Pearson correlation coefficient and p-value of each corresponding parameter. All the Pearson corre-lation coefficients in the table indicated low correcorre-lations, which were 0.1 to 0.3 or -0.1 to -0.3. Although the results were not significant, we could infer from the table that the length of the day positively affected the willingness of Flickr users to take and upload the photos to Flickr; the percent-age of sunshine duration was the secondary parameter that positively affected the amounts of photos on Flickr. On the other hand, the amount of rainfall has a negative relation-ship with Am, which meant that more the rainfall, the less
(a) queensday
(b) gaypride
Figure 14: Top-30 co-occurrent tags of ”queensday” and ”gaypride”
the photos would be taken/ uploaded on Flickr. In addition, we investigated the effects behind hourly patterns of Flickr photos (see Figure 9). We used the variables of sunrise, so-lar noon, and sunset because their changes were according to the seasons. We recalculated the daily data into average monthly data and visualised it as shown in Figure 15. We found that the trend between solar parameters and hourly patterns of Flickr photos was similar. For example, the num-ber of photos increased after the sunrise and decreased after the sunset by observing both trends in visualisations.
Table 2: Pearson correlation coefficient between Am
and other variables
Independent variables r p-value Temperature µt (◦C) 0.135 0.123
Sunshine duration sm(%) 0.166 0.057
Rainfall rm(mm) -0.045 0.61
Day length lm(minutes) 0.192 0.027
In addition to the natural phenomena, we gathered the in-formation about national holidays in The Netherlands and annual events in Amsterdam. Table 3 shows the number of Dutch national holidays and Amsterdam’s annual events in each month (see details in Appendix, Table A2). Since Easter Day sometimes falls in either March or April, we count the days, which are during Easter’s weekend, as 0.5 days. In general, there are less events and holidays at the beginning than in the other months in a year; April is the month where many events start and the number of events in September begins decreasing until the end of the year. The Pearson correlation coefficient between “monthly Flickr pho-tos” and “Dutch holidays and annual events” is 0.5, which means there exists a high correlation.
5.
DISCUSSION AND CONCLUSION
Figure 15: Average monthly time of sunrise, solar noon, and sunset during 2004 and 2014
Table 3: Number of dutch national holidays and an-nual events in Amsterdam. (*: Average of Good Friday, Easter Sunday, and Easter Monday i.e 3 ÷ 2) Jan Feb Mar Apr May Jun National holidays 1 0 *1.5 *2.5 5 0
Annual events 5 5 4 10 8 11 Jul Aug Sep Oct Nov Dec National holidays 0 0 0 0 0 4
Annual events 20 28 18 13 5 3
5.1
Experimental Results
The purpose of this research was to discover the temporal patterns in the use of Flickr in Amsterdam and predict the possible patterns in the future. Our visualisations and anal-ysis provided the insights in our data.
First of all, by observing the monthly trend of Flickr post-ings, there were more Flickr photos captured in spring and summer in Amsterdam, particularly in April and August. The monthly number of Dutch holidays and events was cor-related to the number of Flickr photos in our dataset (cf. Figure 6), which was verified by a high Pearson correlation coefficient. We can thus predict that the more the holi-days and events are, the more photos will be taken and up-loaded on Flickr. Our second finding is that Flickr users will take/ upload more photos on the weekends than dur-ing the weekdays as we expected. Additional evidence came from assigning tf-idf weighting. The results showed that there were more representative event-related tags, such as “football” related tags, on the weekends than during week-days when observing the data of 2013. Next, the analysis of hourly patterns showed that there were more photos cap-tured between 1 p.m and 4 p.m of a day and then the number of photos increased after the sunrise and decreased after the sunset by observing both trends in Figure 9 and Figure 15. Finally, we analysed two given tags, which were “queens-day” and “gaypride”, in different granularity. As we can see in the visualisations, two events led to a monthly trend as can be expected. However, tag “gaypride” had the peaks on Saturday and during 3 p.m. and 5 p.m.; on the other hand, tag “queensday” occurred constantly and gradually when ob-served as daily patterns and hourly patterns. We conjecture the reason of the results is that Queen’s Day is a national
holiday on 30th April, which means that people will have a day off on this day either it is on the weekday or on the weekend; on the other hand, even though Amsterdam Gay Pride is held between the end of July and the beginning of August, the highlight of Gay Pride, canal parade, normally starts at 2 p.m. until 5 p.m. on the first Saturday in August. For the purpose of finding the correlation between other pa-rameters and the number of Flickr photos in our dataset, we computed Pearson correlation in our experiment. The weather parameters indicated low correlations with Flickr usage patterns, which is not consistent with the previous experiment in [9]. The difference between our results and theirs might be due to the type of activity. Going out to vote on an election day was examined in their case, while by contrast, any kind of activity either indoor or outdoor that had been posted were analysed in our experiment. However, the result showed that the parameter of day length positively affected the willingness of Flickr users to take and upload the photos on Flickr most, while the parameter of sunshine duration was the second. There is a negative correlation be-tween the rainfall and the number of Flickr photos in our dataset.
Above all, we further investigated tag co-occurrence in our research. In top-down method, most of the co-occurring tags in the result were related to the given tag after com-bining the background knowledge of the events. For exam-ple, on Queen’s Day, there were celebrations throughout the Netherlands especially in Amsterdam, which is one of the largest world’s street parties. Since orange is the colour of the Dutch Royal Family, people will wear something orange on this day. In addition, the last Queen’s Day was held in 2013, and the first King’s Day was on April 26th, 2014, one day before Willem-Alexander ’s birthday. The above texts printed in italics are the co-occurring tags identified in our experiment. As a result, we can get an overview of an event by observing the co-occurring tags of a given event-related tag.
5.2
Limitations and Future work
Our research has several limitations. One limitation of our research is that we only investigated textual content instead of combining different modalities, such as visual and text; under these circumstances, the results were not interpretable sometimes. For example, we investigated the tags in July of 2005 in order to find the reason why it exceptionally bursted in comparison to other years. Unfortunately, we could not find the informative tags in this cluster by our experiment. Turning to another experiment dealing with the daily pat-terns in 2013, we found that “gay” related tags are highly tagged on Saturday even though the peak, which is gay canal parade, of Amsterdam Gay Pride is only held on the first Saturday of August. We can only conjecture that Amster-dam Gay canal parade is the most distinctive event on Sat-urday in the whole 2013. Furthermore, the hourly patterns showed few photos captured at night even though nightlife is characteristic in Amsterdam. A possible reason could be that people are not willing to or not able to share the pho-tos of nightlife. However, gathering information from other social media sites can be one of the solutions since differ-ent platforms may behave differdiffer-ently according to the result shown in [16].
In our experiment, we did not take the language transla-tions into account, so some non-English tags could not be analysed properly. For example, several non-English “ams-terdam” tags were regarded as different terms. Also, we did not apply other clustering measures to compare the effec-tiveness of different algorithms. For example, Xu et al. [4] showed an effective result of finding semantic tags related-ness by implementing four co-occurrence measures, namely Jaccard, Overlap, Dice, and PMI. There are many possible future directions for this research. Our experiments were de-signed to discover the temporal patterns in the use of Flickr in Amsterdam. However, it would be interesting to extract geo-spatial information from our dataset so that we can de-tect the flow of activities in a city, as done in previous work [16][14][34]. Another possible improvement could be visual-ising the data in an interactive way, as in the applications shown in [8] and [30], so that the users can interact with the data and get more insights from it. Finally, our ap-proach could be applied to other cities or collections of ob-jects with temporal attributes, such as data from Twitter or Foursquare.
References
[1] matplotlib. URL http://matplotlib.org/.
[2] Morgan Ames and Mor Naaman. Why we tag: moti-vations for annotation in mobile and online media. In Proceedings of the SIGCHI conference on Human fac-tors in computing systems, pages 971–980. ACM, 2007. [3] Grigory Begelman, Philipp Keller, Frank Smadja, et al. Automated tag clustering: Improving search and explo-ration in the tag space. In Collaborative Web Tagging Workshop at WWW2006, Edinburgh, Scotland, pages 15–33, 2006.
[4] Di Cai. An information-theoretic foundation for the measurement of discrimination information. Knowledge and Data Engineering, IEEE Transactions on, 22(9): 1262–1273, 2010.
[5] Ling Chen and Abhishek Roy. Event detection from flickr data through wavelet-based spatial analysis. In Proceedings of the 18th ACM conference on Informa-tion and knowledge management, pages 523–532. ACM, 2009.
[6] Justin Cranshaw, Raz Schwartz, Jason I Hong, and Norman M Sadeh. The livehoods project: Utilizing social media to understand the dynamics of a city. In ICWSM, 2012.
[7] dateandtime.info. URL http://dateandtime.info/ index.php.
[8] Micah Dubinko, Ravi Kumar, Joseph Magnani, Jas-mine Novak, Prabhakar Raghavan, and Andrew Tomkins. Visualizing tags over time. ACM Transac-tions on the Web (TWEB), 1(2):7, 2007.
[9] Rob Eisinga, Manfred Te Grotenhuis, and Ben Pelzer. Weather conditions and voter turnout in dutch national parliament elections, 1971–2010. International journal of biometeorology, 56(4):783–786, 2012.
[10] Darrell Etherington. Flickr at 10: 1m photos shared per day, 1701tb free. URL http://techcrunch.com/ 2014/02/10/flickr-at-10-1m-photos-shared-per-day-170-increase-since-making-1tb-free/. [11] Stephen Few. Data visualization for human perception.
The Encyclopedia of Human-Computer Interaction, 2nd Ed., 2013.
[12] Claudiu S Firan, Mihai Georgescu, Wolfgang Nejdl, and Raluca Paiu. Bringing order to your photos: event-driven classification of flickr images based on social knowledge. In Proceedings of the 19th ACM interna-tional conference on Information and knowledge man-agement, pages 189–198. ACM, 2010.
[13] Royal Netherlands Meteorological Institute. Daily weather data for the netherlands. URL http://www.knmi.nl/climatology/daily_data/ selection.cgi.
[14] Lyndon Kennedy, Mor Naaman, Shane Ahern, Rahul Nair, and Tye Rattenbury. How flickr helps us make sense of the world: context and content in community-contributed media collections. In Proceedings of the 15th international conference on Multimedia, pages 631–640. ACM, 2007.
[15] Lyndon S Kennedy, Shih-Fu Chang, and Igor V Koz-intsev. To search or to label?: predicting the perfor-mance of search-based automatic image classifiers. In Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pages 249–258. ACM, 2006.
[16] Yin-Hsi Kuo, Yan-Ying Chen, Bor-Chun Chen, Wen-Yu Lee, Chun-Che Wu, Chia-Hung Lin, Yu-Lin Hou, Wen-Feng Cheng, Yi-Chih Tsai, Chung-Yen Hung, et al. Discovering the city by mining diverse and multimodal data streams. In Proceedings of the ACM International Conference on Multimedia, pages 201–204. ACM, 2014. [17] Henri Lefebvre. Rhythmanalysis: Space, time and
ev-eryday life. A&C Black, 2004.
[18] Jure Leskovec, Anand Rajaraman, and Jeffrey David Ullman. Mining of massive datasets. Cambridge Uni-versity Press, 2014.
[19] Linna Li, Michael F Goodchild, and Bo Xu. Spatial, temporal, and socioeconomic patterns in the use of twit-ter and flickr. cartography and geographic information science, 40(2):61–77, 2013.
[20] Yingjiu Li, X Sean Wang, and Sushil Jajodia. Discov-ering temporal patterns in multiple granularities. In Temporal, Spatial, and Spatio-Temporal Data Mining, pages 5–19. Springer, 2001.
[21] Plotly Python Library. Plotly. URL https://plot.ly/ python/.
[22] Mary Meeker. 2014 internet trends. Technical report, KPCP, May 28 2014.
[23] Franck Michel. Flickr explorer: What makes ex-plored photos so special? Technical report, 2013. URL https://dl.dropboxusercontent.com/ u/23485314/flickrstats/
What_makes_Explored_photos_so_special.pdf. [24] Statistics Netherlands. Same-sex couples in the
nether-lands. URL http://www.cbs.nl/en-GB/menu/home/ default.htm.
[25] Tye Rattenbury, Nathaniel Good, and Mor Naaman. Towards automatic extraction of event and place se-mantics from flickr tags. In Proceedings of the 30th an-nual international ACM SIGIR conference on Research and development in information retrieval, pages 103– 110. ACM, 2007.
[26] B¨orkur Sigurbj¨ornsson and Roelof Van Zwol. Flickr tag recommendation based on collective knowledge. In Pro-ceedings of the 17th international conference on World Wide Web, pages 327–336. ACM, 2008.
[27] Siiri Silm and Rein Ahas. The temporal variation of ethnic segregation in a city: Evidence from a mobile phone use dataset. Social science research, 47:30–43, 2014.
[28] Edward R Tufte and PR Graves-Morris. The visual display of quantitative information, volume 2. Graphics press Cheshire, CT, 1983.
[29] Jorrit van den Berg. Automatic multimodal summari-sation of touristic routes for interactive city exploration. Master’s thesis, University of Amsterdam, 2015. [30] Taowei David Wang, Catherine Plaisant, Ben
Shnei-derman, Neil Spring, David Roseman, Greg Marchand, Vikramjit Mukherjee, and Mark Smith. Temporal sum-maries: supporting temporal categorical searching, ag-gregation and comparison. Visualization and Com-puter Graphics, IEEE Transactions on, 15(6):1049– 1056, 2009.
[31] Zheng Xu, Xiangfeng Luo, Yunhuai Liu, Lin Mei, and Chuanping Hu. Measuring semantic relatedness be-tween flickr images: from a social tag based view. The Scientific World Journal, 2014, 2014.
[32] Yi Hsuan Yang, Po Tun Wu, Ching Wei Lee, Kuan Hung Lin, Winston H Hsu, and Homer H Chen. Contextseer: context search and recommendation at query time for shared consumer photos. In Proceedings of the 16th ACM international conference on Multime-dia, pages 199–208. ACM, 2008.
[33] Junjie Yao, Bin Cui, Yuxin Huang, and Yanhong Zhou. Bursty event detection from collaborative tags. World Wide Web, 15(2):171–195, 2012.
[34] Haipeng Zhang, Mohammed Korayem, Erkang You, and David J Crandall. Beyond co-occurrence: discov-ering and visualizing tag relationships from geo-spatial and temporal similarities. In Proceedings of the fifth ACM international conference on Web search and data mining, pages 33–42. ACM, 2012.
Table A1: Tag ranking according to tf-idf of weekdays and weekends in 2013. “day 1-4” is a cluster of the days from Monday until Thursday
“day 1-4” (2013) tf-idf Friday (2013) tf-idf Saturday (2013) tf-idf Sunday (2013) tf-idf
1 sonyslta77 0.00163 voetbaltoernooi 0.01288 dag1 0.00455 roda23a5 0.00826
2 sonya77 0.00163 wvhedw 0.01288 gaypride2013 0.00208 roda23 0.00826
3 sonyslta77v 0.00163 voetbalwedstrijd 0.00535 gayparade 0.00189 zondag 0.00782
4 hogekwaliteit 0.00088 fcalmerea1 0.00535 homoemancipatie 0.00189 tournament 0.00781
5 scherpgesteld 0.00084 copy 0.00177 gayliberation 0.00189 voetbalwedstrijd 0.00749
6 neilbirchall 0.0008 toutanchamon 0.00173 botenoptocht 0.00189 fcalmerea1 0.00749
7 rufc 0.00068 copies 0.00173 uitdekast 0.00182 soccermatch 0.00654
8 canaltour 0.00062 reportage 0.00167 outofthecloset 0.00182 footballmatch 0.00654
9 filter=chameleon 0.00055 kunstbende 0.00163 citycenter 0.00143 huawei 0.00654
10 lon=493910551 0.00052 dijkshoorn 0.00102 makassarplein 0.0014 afcnautilus 0.00652
11 lat=5231390974 0.00052 spanning 0.00099 orlyplein 0.00133 soccerteamhuaweiamsterdam 0.0065
12 clubziggo 0.00052 modellen 0.00099 kombuispraat 0.00123 rugby 0.00611
13 ziggolive 0.00052 landelijk 0.00099 zeilwedstrijd 0.00123 fcalmere 0.00477
14 500px 0.00051 karakter 0.00099 sailingevents 0.00123 sunday 0.00325
15 stnicolaaslyceum 0.00045 kenmerken 0.00099 zeilen 0.00123 fcweesp 0.00324
16 topazadjust 0.00043 features 0.00099 canalparade2013 0.00088 sevens 0.00324
17 kermisopdedam 0.0004 beeldverhaal 0.00099 chipcard 0.00078 toernooi 0.00324
18 lightfestival2013 0.00038 jukbeenderen 0.00099 quantifiedselfconf 0.00073 dag2 0.00295
19 topazdenoise 0.00036 cheek 0.00099 muiden 0.00069 usrs 0.00227
20 african 0.00036 verhalend 0.00099 telespy 0.00066 thorsrc 0.00175
21 creativemorning 0.00034 5dm3 0.00099 knzbntc 0.00065 thestudenthotel 0.00168
22 tutankhamunexhibition 0.00032 mannequins 0.00099 tsaarpeterhuisje 0.00064 vetbal 0.00144
23 theatreschool 0.00031 voorbereiding 0.00099 westfriesland 0.00059 castricum 0.00134
24 mergeto32bithdr 0.00031 bones 0.00099 indischebuurt 0.00059 castricumserc 0.00132
25 diamondwheel 0.00031 mannequin 0.00099 amsterdamoost 0.00058 delft 0.00113
26 reuzenrad 0.00031 models 0.00099 opening 0.00058 delftsrc 0.00098
27 photomatix 0.0003 catwalk 0.00099 beaconsfield 0.00054 sonyslta77 0.00096
28 amsterdamrai 0.00029 ontwerper 0.00099 design020 0.00052 sonya77 0.00096
29 omnpro 0.00029 bureauu 0.00099 sailing 0.00051 sonyslta77v 0.00096
Table A2: National holidays in Holland (printed in italics) and annual events in Amsterdam; *: Good Friday, Easter Sunday, and Easter Monday are either in March or April sometimes. **: Official Public Holiday Every 5 Years
January February March
New Year’s Day Amsterdam Salsa Festival *Good Friday
Cross-Linx Festival *Easter Sunday
Flamenco Biennale Dirty Dutch *Easter Monday
Grauzone Festival Flamenco Biennale
Impro Amsterdam Stukafest 5 Days Off
Jumping Amsterdam Cinedans
National Tulip Day Dirty Dutch
Roze Filmdagen
April May June
King’s Day National Remembrance Day Amsterdam Open Air
**Liberation Day Architecture Day Amsterdam
Amsterdam Denim Days Ascension Day Artis Zoomeravonden
Amsterdams Studenten Festival Pentecost Sunday Artzuid
Cinima Arabe Pentecost Monday Awakenings Festival
Cinemasia Film Festival Church Night
Dgtl Festival 909 Festival Defqon.1
Dirty Dutch Artzuid Doek Festival
Imagine Film Festival Bevrijdingspop Mystic Garden Festival
Indomania Chateau Festival On The Roof
Sotu Festival De Zon World Press Photo
World Press Photo Diynamic Festival
Het Vrije Westen World Press Photo
July August September Amsterdam Gay Pride Amsterdam Gay Pride Amsterdam City Swim Amsterdam Live On Stage Amsterdam Maker Festival Amsterdam Fringe Festival Amsterdam Roots Festival Appelsap Fresh Music Festival Amsterdam Jazz Festival Artis Zoomeravonden Artis Zoomeravonden Annual Dutch Art Fair
Artzuid Artzuid Artzuid
Buiten Westen Bijlmerbios Atlas Festival
Comedytrain International Summer Festival Canal Parade Banenbeurs
C.R.A.F.T. Festival Comedytrain International Summer Festival Bollywood At The Park De Wereld Draait Buiten Dance Valley Discovery Festival Dekmantel Festival De Parade Theatre Festival Dutch Theatre Festival
Edelwise Aan Zee Dekmantel Festival International Jewish Music Festival
Electronic Family Dutch Valley Jordaan Festival Kwaku Summer Festival Encore Festival Magneet Festival Landjuweel Festival Gaasper Pleasure Metro Movies Milkshake Festival Grachtenfestival Amsterdam Nationale Orgeldag On The Roof Haarlem Jazz & More Pop Up Week Amsterdam Robeco Summernights Hartjesdagen Schuttersdag
Sensation Junior Grachtenfestival Valtifest Unseen Photo Fair Kwaku Summer Festival
World Press Photo Landjuweel Festival Latin Village Festival Loveland Festival Mysteryland On The Roof
Opera In The Garden Prinsengracht Concert Robeco Summernights Uitmarkt
October November December
Afrovibes Festival Amsterdam Art Weekend Sinterklaas Amsterdam Dance Event Elle Festival Christmas Day Amsterdam Music Festival Food Film Festival Boxing Day Awakenings Ade Special International Storytelling Festival Amsterdam New Year’s Eve Camera Japan Jazzfest Amsterdam
Cello Biennale 538 Jingleball
Cinekid Chamber Music Festival
Dancing On The Edge Tangomagia Xiv Festival Driving North
Indie Kids Mini Festival
Klik! Amsterdam Animation Festival Pint Bokbierfestival
