Category: nerdy


Using Socioviz and Gephi to map the twitterverse

April 9th, 2017 — 5:34pm

For quite some time I have wished that Twitter had a native visualization tool that let you see what was trending in real time. There are lists of course, but I’m a visual person and there is so much potential for information in a good network visualization. I already knew what it should look like, as I’d been making similar visualizations using the free graph visualiser program GePhi for a few years, and I knew I could do it myself but the hurdle for me was always getting hold of the data and processing it – I never had the time with the tools that were then available. Well, I recently discovered Socioviz which makes it really easy to get hold of recent twitter data, searchable and downloadable in a file that can be exported straight into GePhi. With socioviz I have finally been able to make twitter visualisations and here’s what i’ve done so far..

This is an example of what the whole of the activity on twitter looks like, over a couple of seconds, when charted as a network using GePhi.

Here is a close up:

How to read this? All the names are people’s handles (with the @ removed from the front). A line is drawn between them when both handles feature in a single tweet (eg. in a reply or a retweet). If this happens more than once then the line gets thicker and this draws them closer together.

As a handle gets mentioned more and more then it’s dot (called a ‘node’) grows – or there is another way of displaying only those handles which are generating lots of tweets – more on that later.

In the images above you can see that ‘youtube‘ and’situt1011‘ were among the most popular handles on twitter in that instant when i got the data.*  They were obviously being included in a lot of retweets. If you look at the close up, you can see realdonaldtrump is there of course, not necessarily tweeting himself, but being included in other’s tweets (those who fan out around him). There are also a couple of little knotty patches of twitter handles all tweeting each other. Either they are all part of a large reply (now that twitter lets you include dozens of names in replies) that is getting a lot of retweets, or they are all tweeting at each other – behaviour which looks a bit suspiciously bot-driven. The accounts in the knot shown here are not particularly worth visiting.

 

 

If you focus on a keyword, or a hashtag, or a handle, socioviz lets you grab up to 5000 tweets (with an account) which include the string you are looking for. I ran this query on the handles @lindasuhler and @lousisemensch – and amalgamated a few days worth of data in GePhi, to see which user groups these two heavy twitter users had.  As anyone who follows US politics on twitter will know, these two users come from opposite ends of the political spectrum (@lindasuhler is also actually a suspected bot but that’s another story), but what really surprised me when I mapped both of their very large networks (based on a couple of day’s worth of data) is that there is virtually no crossover between them. It seems that apart from some connexions around wikileaks, followers of these two accounts are not interacting online at all. They could be interacting on other topics, which I didn’t check, but not in conversations that involve either @lindasuhler or @lousisemensch.

Because this tool is so fast now, I can do more or less real time analysis of emerging trends on twitter and plot the network to see what is really going on. I’ve done this recently on the #SyriaHoax hashtag which generated attention online as it seemed to come out of nowhere. There are lots of conspiracy theories about not only the ‘hoax’ itself, but also about who is starting these twitter storms – but simply looking at the data, it is easy to see who the main players are:

The first image shows a large chunk of twitter users who tweeted about #SyriaHoax in the days 6-8 April. The larger dots are the users who tweeted most often, led by paulieabeles. Some of these seem to have bot-like behaviour. Note that these users often don’t have many followers, and are often quitre new accounts. This contrasts with the other formations you can see – of a single popular user getting lots of single retweets from different users – which looks like a sort of dandelion seed head in these graphs (in this instance the largest of these in the bottom right corner is Caroline O).

 

By focusing on the most mentioned handles in a hashtag – in the example above I looked at #auspol activity for 4 April 2017 – what you see are the important topics of conversation, rather than busiest tweeters. It draws a different picture, and doesn’t draw out the heavy users, although it does still find out whatever their obsession of the moment is.

I hope this is interesting and if you’d like to see some more recent activity, have a look to see what I might have done on twitter today.

***

*a caveat: actually this wasn’t a complete snapshot, as it only included tweets that included certain english language characters like ‘a’. So it’s not a perfect representation of the whole. But I think it still gives the general idea)

 

Comment » | nerdy, whimsy

Meditations on the primes

January 7th, 2016 — 3:47pm

ONE

Big-Bright-light-psd38714

The first number is really all the numbers. It is the infinite one. Indivisible, or infinitely divisible. One way of conceiving of it is a bright light, streaming in all directions, with no shape and no source. It is also like pure being, pure certainty.

TWO

1775569

By introducing two, we divide the one infinitely into a great grid. We have invented even numbers. Every other even number, half of all the numbers, will be a multiple of two, divorced from the whole.

Two is like a moment of doubt, clouding the certainty that existed with one. It seems to be associated with logic and daemons.

THREE

Dispersive_Prism_Illustration_by_Spigget

Three is the prism. It returns towards one, but two has created the separation. Colours are invented by three, and all the possibilities of aesthetics open up through multiples of three; 9, 12, 15, 18… Three creates many beautiful numbers.

FIVE

jkm120325_854

Five seems to be connected to a lot of earthly life and humanity – the fingers on the hand, echinoderms (but not mollusca which are bilateral) – and it is the first number which seems to hint at the shape of the circle, the wheel.

SEVEN

Piano-keyboard

Seven planets (of antiquity), seven days of the week, seven whole notes in the scale. It is seen as a mystical number, I’m not sure where this comes from, perhaps it doesn’t occur in nature very often.  There are seven systems of symmetry identified in crystals, although no crystals seem to have a symmetry that is septile.

ELEVEN & THIRTEEN & etc

primes_square

I guess here we begin the path that is laid out by the Sieve of Eratosthenes. We seem to be out of our comfort zone with these large primes. Although there is an interesting musical scale using thirteen, but it doesn’t sound very musical for ears of this century.

An here is the interesting question about what is the meaning of the pattern thus described by primes?

No more number 9

On a slightly different topic, I’ve been thinking for a while that the base 10 counting system we use is really unnatural and unsuited for the future. In times to come, I wonder if we will come to prefer binary – some extrapolation of it like octal or hexadecimal.

We are already gaining a new familiarity with numbers like 64,  256 and 512,  thanks to the digital age of computers in which it is natural to count things like megabytes (1024 bytes).

Decimal only came about because of the number of fingers we have – those fleshy growths on your hands.  If we were squids it would certainly be different.

If humanity decides to use hex or octal as our counting system instead of decimal at some point in the future, it will be a bit like how during the last century or so most countries decided to use decimal counting instead of using feet and inches, for simplicity in an increasingly empirical world.

Hex seems the most likely candidate to me because it has an added compactness that comes about from the extra characters, yet there still arent’ too many symbols to remember. But here I think we run into a little connundrum.  Hex when it is used now in computers uses the latin alphabet for numbers 11-16 like thus: 1,2,3,4,5,6,7,8,9,A,B,C,D,E,F.

It could become a little confusing to use e.g. the letter C to refer to 12 when for instance trying to sell eggs.  There may also be a problem during a transition period that that unless your hex number contains a letter it will look like it could be a decimal number.

So because of these problems, we will probably have to completely re-invent the symbols we use for counting. It might be a chance to create an entirely new set of symbols that are neat, efficient, and contain none of the ambiguities that you get from reading the numbers 6 or 9 upside down, or thinking a 1 is an ‘l’ (although it might be hard to change the symbol for ‘1’ as it’s so ubiquitous and commonsense).

The names of numbers might need to be changed too. It makes no sense to be saying ‘teen’ for numbers over 12 when we are not counting in decimal any more. Perhaps it’s another opportunity to borrow some neglected words from old cultures. For instance, here is one idea with some borrowings from Sanskrit..

one 1
two 2
three 3
four 4
five 5
six 6
seven 7
eight 8
nine —
ten — 
eleven —
twelve —
trini (13) —
catur (14)  —
puncha (15)  —
sasa (16) — 10
sasaone (17) — 11
sasatwo (18) — 12

etc..

Of course no-one likes change so the opportunity to introduce this system would probably only occur at the same time as some revolution is occurring, or new civilisation establishing itself and its identity.  Perhaps my great grand children will see to that.

Comments Off on Meditations on the primes | nerdy, whimsy

16 bit decade (nostalgia)

November 1st, 2012 — 5:20pm

Bit of a wander through nostalgia.. here are some images which capture the ’16 bit’ decade for me. The timespan i am thinking of is about 1989-1998, when i first got my teenage hands on computers and was instantly transfixed by the games and the fun in getting into the guts of the OS. The 16-bit era is represented to me by chunky VGA graphics, lots of command line interfaces (MS DOS being ubiquitous), floppy disks, and the birth of internet 1.0.

windows3-1

The somewhat clunky Windows 3.1 – which sat on top of DOS – captures the aesthetic of the 16 bit decade better than any other program I can think of.

prince of persia

Image courtesy mysavepoint.blogspot.com Prince of Persia, had fluid graphics and great music. Those slicing blades made us jump out of our skin every time we mistimed a step.

sq4

Space Quest IV had some memorable scenes which often come back to me at odd moments – the burger joint in the futuristic space arcade, the Latex Babes of Estros (who ignited all sorts of pent up 15yo frustrations), the distopian post apocalyptic landscapes. I often wish i could explore the whole sq4 world instead of just following the narrative mapped out by the game design, it seemed so rich. Image courtesy adventuregamers.com.

doom 1

Doom 1 (courtesy www.flatterco.com) this is the first room in the first level. Doom 1 was a 3d revelation with the sunken floor levels and spooky lighting. The soundtrack had some great tunes too, i think it was E2l5 which was my favourite.

altavista

Meanwhile here is one of the first search engines altavista.. slow and clunky compared to google, still these were the only way into the rapidly growing pile of www pages back in 1995-6. They all proudly reported how many thousands or millions of pages they had found in mere microseconds – pseudo statistics which Google and others continues to give to this day. Image from Lloyd Roberson’s Design Journal (lloydrobersondesignjournal.blogspot.com).

Comments Off on 16 bit decade (nostalgia) | nerdy, whimsy

happy about Higgs

July 4th, 2012 — 7:33am

I am glad that we seem to have found the Higgs particle. Now we can all sleep comfortably with our good looking standard model.

Watching the live feed.. so much data.. fantastic. webcast (will only work for another hour or so)

Comments Off on happy about Higgs | nerdy

Back to top