Since the beginning, Twitter has supported Basic Auth as a form of authentication. On June 30th, however, they plan to pull the plug. If you’re unfamiliar with authentication processes, Basic Auth involves attaching the user’s username and password to a request header. It’s as simple as can be for developers, but not exactly safe for users. Although many developers wouldn’t even consider tampering with a user’s account, the dark side still lingers.

Because of this, OAuth was introduced to keep users’ passwords out of 3rd party hands. Instead, an access token is given to the 3rd party for each user. With it, they can call any of the methods in the API that Basic Auth could, but users can feel safe knowing only Twitter holds their passwords. Though this sounds all well and good, it certainly has its downsides.

For one, the login process isn’t as smooth—for both the user and the developer. This diagram compares the user experience of Basic Auth versus OAuth. As you can see, OAuth is a bit long-winded. It’s even worse for the 3rd party. They bear the burden of encoding and signature parsing. From a user experience standpoint, the worst part of the OAuth process lies in the last three steps. Twitter is not a bank. Requiring the user to copy, paste, and authenticate a pin adds three unnecessary steps.

Prior to DestroyTwitter, I developed DestroyFlickr, which authenticates with Flickr to access its API. This process involves entering a username in the app, which opens Flickr.com, asks you to login if you haven’t already, then verifies that you’d like to authorize this app. Finally, return to the app and you’re good to go.

Now, Twitter does have an alternative authentication method—xAuth. Imagine if Basic Auth and OAuth had a baby. The process is the same for the user as Basic Auth, but the 3rd party is given an access token, just like with OAuth. There’s one problem—it still divulges the user’s password to the 3rd party, just like Basic Auth. Even though Twitter says, “Storage of Twitter usernames and passwords is forbidden,” this single sentence isn’t going to stop a malicious 3rd party from exploiting users.

Let’s look at one last issue that arises with OAuth. For Twitter app developers, how do you authenticate with services like Twitpic?—with the user’s username and password. If the 3rd party can’t store users’ credentials, it’s impossible for them to authenticate with the service. Unless Twitter releases this proposed echo method with enough time to implement, you will either see a massive drop in Twitter service usage or Twitter app developers will ignore the no-storage rule and put us back at square one.

Keep in mind, a lot can change between now and June. Let’s hope Twitter improves the user experience by removing the need for a pin. Let’s look forward to a way to communicate with services without sharing users’ credentials. With enough of a heads-up, June 30th will just be another day. If, however, Twitter makes a change in the eleventh hour, we might see a number of frantic developers.

[update] – Twitter updates OAuth docs to prepare developers for June.

Twitter’s new retweet system isn’t exactly new anymore, but the way in which it is implemented feels like it would be an older system, reducing ease of use and accessibility. Prior to the ‘upgrade’, retweeting consisted of a simple tweet including the original tweet, tweeter and a marker. The marker first started as ‘RT’, but over time, variations were ‘invented.’ In the early DestroyTwiter days, Josh Corliss suggested using ‘>’ instead of ‘RT’, which saves a character and simply looks better. Many users reference tweets with ‘(via @username)’ or ‘/via @username’ to summarize instead of using word-for-word retweets. All of these options can be edited, daisy-chained, and commented on. They also appear in the retweeted user’s mentions timeline, notifying him/her of a retweet. With the new system, all of these features are either diminished or elongated.

To give an idea of the work on the developer, the new retweet system introduces the following API methods:

1
2
3
4
5
6
7

statuses/retweet
statuses/retweets
statuses/id/retweeted_by
statuses/id/retweeted_by/ids
statuses/retweeted_by_me
statuses/retweeted_to_me
statuses/retweeted_of_me

The original system requires only two:

1
2

statuses/update
statuses/mentions

Of course, the new system brings more control, but is it all necessary? In order to see which tweets of yours were retweeted and by whom, the statuses/retweeted_of_me method must be called, followed by the statuses/retweets method for each tweet. Unless you’re whitelisted (have 20,000 API calls), calling these methods, in addition to the common methods, could certainly add up quick. On top of that, both methods would need to be polled consistently to stay up-to-date in case additional retweets were made. Unless the developer of the app you use wants to bend over backwards to notify you of new retweets, you’re forced to visit Twitter.com to check.

Now, is it really such a big deal to know how many times or by whom your tweets are retweeted? Not entirely, but I personally gauge which tweets my followers react to the most by how many times the tweets are retweeted. If I tweet about DestroyTwitter and it’s retweeted a dozen times, then I tweet a video of my cat that only results in crickets, I’ll know to lay off the cat videos—but that’s just me. Other users find new people to follow by seeing who retweets their tweets.

With the new system, Twitter makes it harder on both developers and users. The original system worked and worked well. Many users, like myself, reject the new system and continue to use the original one. Because of this, users must reference multiple locations instead of one. This once simple-as-can-be feature is now a nuisance.

Twitter used to be simple. It still can be depending on which app you use, but if other features are ‘improved upon’ as retweets were, we might end up with unavoidable obstacles. I’m growing tired of services that start out beautifully, then take a turn for the worst after being packed full of unneeded additions. A prime example is Facebook, but that’s for another time.

(photo property of DreamWorks and Universal Studios)

Working with an API can be a hassle—especially if you have no control over it. Twitter’s API has its fair share of obstacles, one of which is paging. When you first look at a method in the Twitter API, you will most likely go straight to the page parameter. It’s simple and expected—page 1 is the most recent and the page number increases as you navigate back in time. This seems like the right solution, but it’s not. The page parameter has a major downside.

Since page 1 is the most recent, the entire paging system is relative and forever changing with every new tweet. This means if your user pages back in time, as new tweets arrive in the Twitter database (for his stream), he’ll see the same tweets finding their way into the next page. If the user pages older once and 20 new tweets arrive in Twitter database, he will see the exact same page the next time he pages older (if the count parameter is set to 20).

This issue can be easily dodged by using the max_id parameter. With it, you can set the maximum tweet id number to appear in the stream, excluding all tweets that appear after it. This can be used in paging by setting max_id to one less than the id of the bottom tweet in the stream. There are, however, a few downsides. Since there is no min_id parameter, you need to store all of the previous max_id parameters, so you can find your way back (assuming you run your app off the raw API data and not a local database). Also, I’ve noticed Twitter caps the number of tweets you can retrieve to about 700 or 2 months (I’m not sure if this is a count-based limit or date-based). This is interesting, considering Twitter indicates in the API docs that they limit the number of retrievable to 3,200. They do note this under “pagination limits” using the page and count parameters, but I assumed we would see this same amount for max_id.

Now, another issue with Twitter pagination, and the original reason I decided to write this post, is gaps in the Twitter stream. If your Twitter app caches tweets to a local database, this is for you. Let’s say someone uses your application once, then takes a day off and returns the next day. In that interim, thousands upon thousands of tweets entered the Twitter database, and possibly your user’s stream. The next time this user starts the application, it loads in the latest tweets, but there is still a slew of tweets from the previous day. What do you do?

Use a combination of the max_id and since_id parameters. Start loading older pages using max_id and when the return is empty, you’ve either caught up to the most recent tweet id from the previous session or you reached Twitter’s limit.

A few of Twitter’s newer methods use cursors for paging, where each result includes two numbers that are used to navigate to the newer or older page. I personally haven’t used any of these methods, so it’s unclear to me whether the numbers are relative or absolute—hopefully, the latter. In any case, I look forward to this implementation in some of the older methods like favorites that doesn’t even have since_id or max_id yet.

A couple months ago, I started working on a Twitter component for my current project at Adobe. I went into this knowing I’d have to finally face the beast… OAuth. Just about every well-known Twitter client out there uses Basic Auth—and for a reason. It’s easy, what the user expects, and gives your app more credibility—there’s no requirement to leave to authenticate through the browser like with OAuth.

About five or six months ago, Twitter decided to enforce the transition. From then on, any application that uses the API must use OAuth in order to see “via [your app]” on tweets published with it—otherwise, it would display “via API.” Since “via” is where apps get probably 90% of their referrals, this was a big deal. Luckily for me, DestroyTwitter existed before that time and Twitter decided not to push the change on the veteran apps. Recently, however, the bad news spread that Basic Auth would be deprecated in June. This means every Twitter app must transition to the pain that is OAuth.

After developing the MAX Companion this past fall and now the more generalized version, I found myself rewriting the Twitter component each time. After a while, the Twitter API code I wrote for DestroyTwitter began to merge with the actual implementation, so it was no longer a standalone library that could easily be used by other projects. These past few months, I’ve been learning a great deal about framework architecture and design patterns. It has led me to realize I need to start fresh.

With all that being said, I’d like introduce a library I started working on two days ago. I call it TwitterAspirin. It’s an AS3 Twitter API library that eases the pain, providing developers with a very powerful tool for communicating with Twitter. Though it’s still a newborn at the moment, I see potential already. The library is built on RobotLegs and uses AS3 Signals instead of events. It’s hosted on GitHub, so the source code will always be available to the public. And, after last night’s commit, its OAuth functionality is complete. Here’s how to use it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

package {
	import com.destroytoday.twitteraspirin.Twitter;
 
	import flash.display.Sprite;
 
	public class Test extends Sprite {
		// set application consumer key and secret
		public var twitter:Twitter = new Twitter(consumerKey, consumerSecret);
 
		public function Test() {
			// add signal listeners
			twitter.oauth.requestTokenSignal.add(requestTokenHandler);
			twitter.oauth.accessTokenSignal.add(accessTokenHandler);
			twitter.oauth.verifyAccessTokenSignal.add(verifyAccessTokenHandler);
		}
 
		// click the 'Authorize' button to get the request token
		protected function authorizeClickHandler():void {
			twitter.oauth.getRequestToken();
		}
 
		// upon receiving the request token, open Twitter in the browser to authorize
		protected function requestTokenHandler(oauth:OAuth, token:OAuthToken):void {
			navigateToURL(new URLRequest(oauth.getAuthorizeURL()));
		}
 
		// return with the provided pin and click the 'Activate' button to get the access token
		protected function activateClickHandler():void {
			twitter.oauth.getAccessToken(pin);
		}
 
		// upon receiving the access token, verify it
		protected function accessTokenHandler(oauth:OAuth, token:OAuthToken):void {
			oauth.verifyAccessToken(token);
		}
 
		// done
		protected function verifyAccessTokenHandler(oauth:OAuth, token:OAuthToken):void {
		}
	}
}

As you can see, it’s extremely easy to use. Not only that, it provides great flexibility. Many libraries are simple to implement, but don’t allow the developer access to certain aspects of the process. With TwitterAspirin, each method returns the loader involved with the operation, giving developers the ability to listen for errors, cancel the operation, or attain the raw API data. The library also uses loader pools to recycle instances, so you can submit a tweet and, while it’s loading, submit another—you don’t have to wait for the first operation to finish. Then, once the operation is complete, the loader is disposed and returned to the pool, which optimizes performance and memory usage.

I’m really excited to see where TwitterAspirin goes and I have nothing but great expectations. Be sure to follow along with development and fork whenever you like.

After asking for access to the Twitter Lists API a number of times, I was first turned down and now ignored. Since DestroyTwitter 2.0 is starting to take shape, I want to make sure it includes Lists, but the lack of API access is a big roadblock. Since Twitter stopped listening, I decided to do a bit of reverse engineering to find out what the available methods are. Here is what I found:

https://twitter.com/[username]/lists.xml

This method returns an array of the specified user’s lists. It includes details about each list (id, name, subscribers, etc) and alsothe return values for the user who created this list. The method utilizes the new cursor format for paging. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

<?xml version="1.0" encoding="UTF-8"?>
<lists_list>
<lists type="array">
<list>
  <id>640968</id>
  <name>Developers</name>
  <full_name>@destroytoday/developers</full_name>
  <slug>developers</slug>
  <subscriber_count>0</subscriber_count>
  <member_count>22</member_count>
  <uri>/destroytoday/developers</uri>
  <mode>private</mode>
  <user>
    <id>14839458</id>
    <name>Jonnie Hallman</name>
    <screen_name>destroytoday</screen_name>
    <location>Baltimore, MD</location>
    <description>Founder of Destroy Today. Developer of DestroyFlickr and DestroyTwitter. Experience Designer at Adobe.</description>
    <profile_image_url>http://a3.twimg.com/profile_images/124402983/about-1_normal.jpg</profile_image_url>
    <url>http://www.destroytoday.com</url>
    <protected>false</protected>
    <followers_count>3328</followers_count>
    <profile_background_color>222222</profile_background_color>
    <profile_text_color>AAADB6</profile_text_color>
    <profile_link_color>00728F</profile_link_color>
    <profile_sidebar_fill_color>222222</profile_sidebar_fill_color>
    <profile_sidebar_border_color>222222</profile_sidebar_border_color>
    <friends_count>129</friends_count>
    <created_at>Tue May 20 00:08:47 +0000 2008</created_at>
    <favourites_count>122</favourites_count>
    <utc_offset>-18000</utc_offset>
    <time_zone>Eastern Time (US &amp; Canada)</time_zone>
    <profile_background_image_url>http://s.twimg.com/a/1257191498/images/themes/theme1/bg.png</profile_background_image_url>
    <profile_background_tile>false</profile_background_tile>
    <statuses_count>12148</statuses_count>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
    <following>false</following>
  </user>
</list>
</lists>
<next_cursor>0</next_cursor>
<previous_cursor>0</previous_cursor>
</lists_list>

https://twitter.com/[username]/lists/subscriptions.xml

This method is the same setup as the previous method in that it returns an array of lists, but this is for lists that the user is following besides his own. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

<?xml version="1.0" encoding="UTF-8"?>
<lists_list>
<lists type="array">
<list>
  <id>1142867</id>
  <name>Flash devs not to follow</name>
  <full_name>@TheFlashBum/flash-devs-not-to-follow</full_name>
  <slug>flash-devs-not-to-follow</slug>
  <subscriber_count>1</subscriber_count>
 
  <member_count>8</member_count>
  <uri>/TheFlashBum/flash-devs-not-to-follow</uri>
  <mode>public</mode>
  <user>
    <id>26755983</id>
    <name>Jesse Freeman</name>
 
    <screen_name>TheFlashBum</screen_name>
    <location>NYC</location>
    <description>I am a homeless Flash Developer</description>
    <profile_image_url>http://a1.twimg.com/profile_images/475933036/flashbum_bio_normal.jpg</profile_image_url>
    <url>http://flashbum.com</url>
    <protected>false</protected>
 
    <followers_count>1146</followers_count>
    <profile_background_color>1A1B1F</profile_background_color>
    <profile_text_color>666666</profile_text_color>
    <profile_link_color>2FC2EF</profile_link_color>
    <profile_sidebar_fill_color>252429</profile_sidebar_fill_color>
    <profile_sidebar_border_color>181A1E</profile_sidebar_border_color>
 
    <friends_count>198</friends_count>
    <created_at>Thu Mar 26 14:07:20 +0000 2009</created_at>
    <favourites_count>140</favourites_count>
    <utc_offset>-18000</utc_offset>
    <time_zone>Eastern Time (US &amp; Canada)</time_zone>
 
    <profile_background_image_url>http://s.twimg.com/a/1257210731/images/themes/theme9/bg.gif</profile_background_image_url>
    <profile_background_tile>false</profile_background_tile>
    <statuses_count>4186</statuses_count>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
 
    <following>true</following>
  </user>
</list>
</lists>
<next_cursor>0</next_cursor>
<previous_cursor>0</previous_cursor>
</lists_list>

https://twitter.com/[username]/lists/[listname]/statuses.xml

This is the statuses method for a specific list. This is the same as any other statuses method. It includes the count, page, max_id, and since_id parameters. I haven’t tested for any others than those. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

<?xml version="1.0" encoding="UTF-8"?>
<statuses type="array">
<status>
  <created_at>Tue Nov 03 23:35:59 +0000 2009</created_at>
  <id>5405262473</id>
  <text>AT&amp;T complaining about Verizon ad accurately portraying the piss-poor suckiness of their service in 98% of the country: http://bit.ly/e4ieH</text>
  <source>&lt;a href=&quot;http://destroytwitter.com/&quot; rel=&quot;nofollow&quot;&gt;DestroyTwitter&lt;/a&gt;</source>
  <truncated>false</truncated>
  <in_reply_to_status_id></in_reply_to_status_id>
  <in_reply_to_user_id></in_reply_to_user_id>
  <favorited>false</favorited>
  <in_reply_to_screen_name></in_reply_to_screen_name>
  <user>
    <id>14551527</id>
    <name>rjowen</name>
    <screen_name>rjowen</screen_name>
    <location>Denver, CO</location>
    <description></description>
    <profile_image_url>http://a3.twimg.com/profile_images/494903369/twitter_normal.jpg</profile_image_url>
    <url>http://rjria.blogspot.com</url>
    <protected>false</protected>
    <followers_count>318</followers_count>
    <profile_background_color>1A1B1F</profile_background_color>
    <profile_text_color>666666</profile_text_color>
    <profile_link_color>2FC2EF</profile_link_color>
    <profile_sidebar_fill_color>252429</profile_sidebar_fill_color>
    <profile_sidebar_border_color>181A1E</profile_sidebar_border_color>
    <friends_count>176</friends_count>
    <created_at>Sat Apr 26 21:06:42 +0000 2008</created_at>
    <favourites_count>8</favourites_count>
    <utc_offset>-25200</utc_offset>
    <time_zone>Mountain Time (US &amp; Canada)</time_zone>
    <profile_background_image_url>http://a1.twimg.com/profile_background_images/4099338/IMG_0224.JPG</profile_background_image_url>
    <profile_background_tile>false</profile_background_tile>
    <statuses_count>1801</statuses_count>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
    <following>false</following>
  </user>
  <geo/>
</status>

https://twitter.com/[username]/lists/[listname]/members.xml

Lastly, the method for adding a user to a list. It looks like a user can be added simply by including the user’s id in the id parameter. The result is the same as lists.xml. This method in particular requires the POST method. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

<?xml version="1.0" encoding="UTF-8"?>
<list>
  <id>640968</id>
  <name>Developers</name>
  <full_name>@destroytoday/developers</full_name>
  <slug>developers</slug>
  <subscriber_count>0</subscriber_count>
  <member_count>22</member_count>
  <uri>/destroytoday/developers</uri>
  <mode>private</mode>
  <user>
    <id>14839458</id>
    <name>Jonnie Hallman</name>
    <screen_name>destroytoday</screen_name>
    <location>Baltimore, MD</location>
    <description>Founder of Destroy Today. Developer of DestroyFlickr and DestroyTwitter. Experience Designer at Adobe.</description>
    <profile_image_url>http://a3.twimg.com/profile_images/124402983/about-1_normal.jpg</profile_image_url>
    <url>http://www.destroytoday.com</url>
    <protected>false</protected>
    <followers_count>3330</followers_count>
    <profile_background_color>222222</profile_background_color>
    <profile_text_color>AAADB6</profile_text_color>
    <profile_link_color>00728F</profile_link_color>
    <profile_sidebar_fill_color>222222</profile_sidebar_fill_color>
    <profile_sidebar_border_color>222222</profile_sidebar_border_color>
    <friends_count>129</friends_count>
    <created_at>Tue May 20 00:08:47 +0000 2008</created_at>
    <favourites_count>122</favourites_count>
    <utc_offset>-18000</utc_offset>
    <time_zone>Eastern Time (US &amp; Canada)</time_zone>
    <profile_background_image_url>http://s.twimg.com/a/1257288876/images/themes/theme1/bg.png</profile_background_image_url>
    <profile_background_tile>false</profile_background_tile>
    <statuses_count>12159</statuses_count>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
    <following>false</following>
  </user>
</list>

In conclusion, I must say it’s pretty sad that an experienced Twitter developer needs to dig through the accepted developers’ work to be able to use the same features. I know Lists are new, but DestroyTwitter has been out far longer than the app I got this from. Also, the retweet and geo-location APIs have been public for months now, though the features aren’t completely implemented. DestroyTwitter doesn’t have millions of dollars worth of funding to bring to the table, but it has a pretty passionate developer who wants the latest features for his users.

[update] Thanks to Lim CHee Aun, who referred me to the draft documentation for Lists. It reveals a few more methods, including the creation and destruction of lists. The API URLs differ quite a bit from the Twitter API, using POST, DELETE, and GET with the same URL to create, delete, and return a list respectively. In the Twitter API, create and destroy are included in the URL itself. I’m hoping for the ability to bulk add/remove users to/from lists. That would seriously alleviate pressure from the hundreds of calls it would take otherwise.

DestroyTwitter is finally on Twitter. For the past nine months, I’ve used my personal Twitter account (@destroytoday) to post news and respond to feedback. That’s all fine by me, but I figured an account of its own would make things more official. Lots of people asked how I was able to get an account with the word “Twitter” in the name. In all honesty, I simply asked. Since DestroyTwitter has been around for a while (for a Twitter app), they were able to make an exception. If you haven’t done so already, go ahead and follow the @destroytwitter account—and keep an eye out over the next few weeks.

Christmas comes early as Twitter ups the API limit to 150 from 100. What does this mean?—more frequent canvas refreshes for users and a little breathing room for developers. I’m still holding out for unlimited API calls, but I know that will always be a dream.