Working with an API can be a hassle—especially if you have no control over it. Twitter’s API has its fair share of obstacles, one of which is paging. When you first look at a method in the Twitter API, you will most likely go straight to the page parameter. It’s simple and expected—page 1 is the most recent and the page number increases as you navigate back in time. This seems like the right solution, but it’s not. The page parameter has a major downside.

Since page 1 is the most recent, the entire paging system is relative and forever changing with every new tweet. This means if your user pages back in time, as new tweets arrive in the Twitter database (for his stream), he’ll see the same tweets finding their way into the next page. If the user pages older once and 20 new tweets arrive in Twitter database, he will see the exact same page the next time he pages older (if the count parameter is set to 20).

This issue can be easily dodged by using the max_id parameter. With it, you can set the maximum tweet id number to appear in the stream, excluding all tweets that appear after it. This can be used in paging by setting max_id to one less than the id of the bottom tweet in the stream. There are, however, a few downsides. Since there is no min_id parameter, you need to store all of the previous max_id parameters, so you can find your way back (assuming you run your app off the raw API data and not a local database). Also, I’ve noticed Twitter caps the number of tweets you can retrieve to about 700 or 2 months (I’m not sure if this is a count-based limit or date-based). This is interesting, considering Twitter indicates in the API docs that they limit the number of retrievable to 3,200. They do note this under “pagination limits” using the page and count parameters, but I assumed we would see this same amount for max_id.

Now, another issue with Twitter pagination, and the original reason I decided to write this post, is gaps in the Twitter stream. If your Twitter app caches tweets to a local database, this is for you. Let’s say someone uses your application once, then takes a day off and returns the next day. In that interim, thousands upon thousands of tweets entered the Twitter database, and possibly your user’s stream. The next time this user starts the application, it loads in the latest tweets, but there is still a slew of tweets from the previous day. What do you do?

Use a combination of the max_id and since_id parameters. Start loading older pages using max_id and when the return is empty, you’ve either caught up to the most recent tweet id from the previous session or you reached Twitter’s limit.

A few of Twitter’s newer methods use cursors for paging, where each result includes two numbers that are used to navigate to the newer or older page. I personally haven’t used any of these methods, so it’s unclear to me whether the numbers are relative or absolute—hopefully, the latter. In any case, I look forward to this implementation in some of the older methods like favorites that doesn’t even have since_id or max_id yet.

A couple months ago, I started working on a Twitter component for my current project at Adobe. I went into this knowing I’d have to finally face the beast… OAuth. Just about every well-known Twitter client out there uses Basic Auth—and for a reason. It’s easy, what the user expects, and gives your app more credibility—there’s no requirement to leave to authenticate through the browser like with OAuth.

About five or six months ago, Twitter decided to enforce the transition. From then on, any application that uses the API must use OAuth in order to see “via [your app]” on tweets published with it—otherwise, it would display “via API.” Since “via” is where apps get probably 90% of their referrals, this was a big deal. Luckily for me, DestroyTwitter existed before that time and Twitter decided not to push the change on the veteran apps. Recently, however, the bad news spread that Basic Auth would be deprecated in June. This means every Twitter app must transition to the pain that is OAuth.

After developing the MAX Companion this past fall and now the more generalized version, I found myself rewriting the Twitter component each time. After a while, the Twitter API code I wrote for DestroyTwitter began to merge with the actual implementation, so it was no longer a standalone library that could easily be used by other projects. These past few months, I’ve been learning a great deal about framework architecture and design patterns. It has led me to realize I need to start fresh.

With all that being said, I’d like introduce a library I started working on two days ago. I call it TwitterAspirin. It’s an AS3 Twitter API library that eases the pain, providing developers with a very powerful tool for communicating with Twitter. Though it’s still a newborn at the moment, I see potential already. The library is built on RobotLegs and uses AS3 Signals instead of events. It’s hosted on GitHub, so the source code will always be available to the public. And, after last night’s commit, its OAuth functionality is complete. Here’s how to use it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

package {
	import com.destroytoday.twitteraspirin.Twitter;
 
	import flash.display.Sprite;
 
	public class Test extends Sprite {
		// set application consumer key and secret
		public var twitter:Twitter = new Twitter(consumerKey, consumerSecret);
 
		public function Test() {
			// add signal listeners
			twitter.oauth.requestTokenSignal.add(requestTokenHandler);
			twitter.oauth.accessTokenSignal.add(accessTokenHandler);
			twitter.oauth.verifyAccessTokenSignal.add(verifyAccessTokenHandler);
		}
 
		// click the 'Authorize' button to get the request token
		protected function authorizeClickHandler():void {
			twitter.oauth.getRequestToken();
		}
 
		// upon receiving the request token, open Twitter in the browser to authorize
		protected function requestTokenHandler(oauth:OAuth, token:OAuthToken):void {
			navigateToURL(new URLRequest(oauth.getAuthorizeURL()));
		}
 
		// return with the provided pin and click the 'Activate' button to get the access token
		protected function activateClickHandler():void {
			twitter.oauth.getAccessToken(pin);
		}
 
		// upon receiving the access token, verify it
		protected function accessTokenHandler(oauth:OAuth, token:OAuthToken):void {
			oauth.verifyAccessToken(token);
		}
 
		// done
		protected function verifyAccessTokenHandler(oauth:OAuth, token:OAuthToken):void {
		}
	}
}

As you can see, it’s extremely easy to use. Not only that, it provides great flexibility. Many libraries are simple to implement, but don’t allow the developer access to certain aspects of the process. With TwitterAspirin, each method returns the loader involved with the operation, giving developers the ability to listen for errors, cancel the operation, or attain the raw API data. The library also uses loader pools to recycle instances, so you can submit a tweet and, while it’s loading, submit another—you don’t have to wait for the first operation to finish. Then, once the operation is complete, the loader is disposed and returned to the pool, which optimizes performance and memory usage.

I’m really excited to see where TwitterAspirin goes and I have nothing but great expectations. Be sure to follow along with development and fork whenever you like.

After asking for access to the Twitter Lists API a number of times, I was first turned down and now ignored. Since DestroyTwitter 2.0 is starting to take shape, I want to make sure it includes Lists, but the lack of API access is a big roadblock. Since Twitter stopped listening, I decided to do a bit of reverse engineering to find out what the available methods are. Here is what I found:

https://twitter.com/[username]/lists.xml

This method returns an array of the specified user’s lists. It includes details about each list (id, name, subscribers, etc) and alsothe return values for the user who created this list. The method utilizes the new cursor format for paging. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

<?xml version="1.0" encoding="UTF-8"?>
<lists_list>
<lists type="array">
<list>
  <id>640968</id>
  <name>Developers</name>
  <full_name>@destroytoday/developers</full_name>
  <slug>developers</slug>
  <subscriber_count>0</subscriber_count>
  <member_count>22</member_count>
  <uri>/destroytoday/developers</uri>
  <mode>private</mode>
  <user>
    <id>14839458</id>
    <name>Jonnie Hallman</name>
    <screen_name>destroytoday</screen_name>
    <location>Baltimore, MD</location>
    <description>Founder of Destroy Today. Developer of DestroyFlickr and DestroyTwitter. Experience Designer at Adobe.</description>
    <profile_image_url>http://a3.twimg.com/profile_images/124402983/about-1_normal.jpg</profile_image_url>
    <url>http://www.destroytoday.com</url>
    <protected>false</protected>
    <followers_count>3328</followers_count>
    <profile_background_color>222222</profile_background_color>
    <profile_text_color>AAADB6</profile_text_color>
    <profile_link_color>00728F</profile_link_color>
    <profile_sidebar_fill_color>222222</profile_sidebar_fill_color>
    <profile_sidebar_border_color>222222</profile_sidebar_border_color>
    <friends_count>129</friends_count>
    <created_at>Tue May 20 00:08:47 +0000 2008</created_at>
    <favourites_count>122</favourites_count>
    <utc_offset>-18000</utc_offset>
    <time_zone>Eastern Time (US &amp; Canada)</time_zone>
    <profile_background_image_url>http://s.twimg.com/a/1257191498/images/themes/theme1/bg.png</profile_background_image_url>
    <profile_background_tile>false</profile_background_tile>
    <statuses_count>12148</statuses_count>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
    <following>false</following>
  </user>
</list>
</lists>
<next_cursor>0</next_cursor>
<previous_cursor>0</previous_cursor>
</lists_list>

https://twitter.com/[username]/lists/subscriptions.xml

This method is the same setup as the previous method in that it returns an array of lists, but this is for lists that the user is following besides his own. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

<?xml version="1.0" encoding="UTF-8"?>
<lists_list>
<lists type="array">
<list>
  <id>1142867</id>
  <name>Flash devs not to follow</name>
  <full_name>@TheFlashBum/flash-devs-not-to-follow</full_name>
  <slug>flash-devs-not-to-follow</slug>
  <subscriber_count>1</subscriber_count>
 
  <member_count>8</member_count>
  <uri>/TheFlashBum/flash-devs-not-to-follow</uri>
  <mode>public</mode>
  <user>
    <id>26755983</id>
    <name>Jesse Freeman</name>
 
    <screen_name>TheFlashBum</screen_name>
    <location>NYC</location>
    <description>I am a homeless Flash Developer</description>
    <profile_image_url>http://a1.twimg.com/profile_images/475933036/flashbum_bio_normal.jpg</profile_image_url>
    <url>http://flashbum.com</url>
    <protected>false</protected>
 
    <followers_count>1146</followers_count>
    <profile_background_color>1A1B1F</profile_background_color>
    <profile_text_color>666666</profile_text_color>
    <profile_link_color>2FC2EF</profile_link_color>
    <profile_sidebar_fill_color>252429</profile_sidebar_fill_color>
    <profile_sidebar_border_color>181A1E</profile_sidebar_border_color>
 
    <friends_count>198</friends_count>
    <created_at>Thu Mar 26 14:07:20 +0000 2009</created_at>
    <favourites_count>140</favourites_count>
    <utc_offset>-18000</utc_offset>
    <time_zone>Eastern Time (US &amp; Canada)</time_zone>
 
    <profile_background_image_url>http://s.twimg.com/a/1257210731/images/themes/theme9/bg.gif</profile_background_image_url>
    <profile_background_tile>false</profile_background_tile>
    <statuses_count>4186</statuses_count>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
 
    <following>true</following>
  </user>
</list>
</lists>
<next_cursor>0</next_cursor>
<previous_cursor>0</previous_cursor>
</lists_list>

https://twitter.com/[username]/lists/[listname]/statuses.xml

This is the statuses method for a specific list. This is the same as any other statuses method. It includes the count, page, max_id, and since_id parameters. I haven’t tested for any others than those. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

<?xml version="1.0" encoding="UTF-8"?>
<statuses type="array">
<status>
  <created_at>Tue Nov 03 23:35:59 +0000 2009</created_at>
  <id>5405262473</id>
  <text>AT&amp;T complaining about Verizon ad accurately portraying the piss-poor suckiness of their service in 98% of the country: http://bit.ly/e4ieH</text>
  <source>&lt;a href=&quot;http://destroytwitter.com/&quot; rel=&quot;nofollow&quot;&gt;DestroyTwitter&lt;/a&gt;</source>
  <truncated>false</truncated>
  <in_reply_to_status_id></in_reply_to_status_id>
  <in_reply_to_user_id></in_reply_to_user_id>
  <favorited>false</favorited>
  <in_reply_to_screen_name></in_reply_to_screen_name>
  <user>
    <id>14551527</id>
    <name>rjowen</name>
    <screen_name>rjowen</screen_name>
    <location>Denver, CO</location>
    <description></description>
    <profile_image_url>http://a3.twimg.com/profile_images/494903369/twitter_normal.jpg</profile_image_url>
    <url>http://rjria.blogspot.com</url>
    <protected>false</protected>
    <followers_count>318</followers_count>
    <profile_background_color>1A1B1F</profile_background_color>
    <profile_text_color>666666</profile_text_color>
    <profile_link_color>2FC2EF</profile_link_color>
    <profile_sidebar_fill_color>252429</profile_sidebar_fill_color>
    <profile_sidebar_border_color>181A1E</profile_sidebar_border_color>
    <friends_count>176</friends_count>
    <created_at>Sat Apr 26 21:06:42 +0000 2008</created_at>
    <favourites_count>8</favourites_count>
    <utc_offset>-25200</utc_offset>
    <time_zone>Mountain Time (US &amp; Canada)</time_zone>
    <profile_background_image_url>http://a1.twimg.com/profile_background_images/4099338/IMG_0224.JPG</profile_background_image_url>
    <profile_background_tile>false</profile_background_tile>
    <statuses_count>1801</statuses_count>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
    <following>false</following>
  </user>
  <geo/>
</status>

https://twitter.com/[username]/lists/[listname]/members.xml

Lastly, the method for adding a user to a list. It looks like a user can be added simply by including the user’s id in the id parameter. The result is the same as lists.xml. This method in particular requires the POST method. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

<?xml version="1.0" encoding="UTF-8"?>
<list>
  <id>640968</id>
  <name>Developers</name>
  <full_name>@destroytoday/developers</full_name>
  <slug>developers</slug>
  <subscriber_count>0</subscriber_count>
  <member_count>22</member_count>
  <uri>/destroytoday/developers</uri>
  <mode>private</mode>
  <user>
    <id>14839458</id>
    <name>Jonnie Hallman</name>
    <screen_name>destroytoday</screen_name>
    <location>Baltimore, MD</location>
    <description>Founder of Destroy Today. Developer of DestroyFlickr and DestroyTwitter. Experience Designer at Adobe.</description>
    <profile_image_url>http://a3.twimg.com/profile_images/124402983/about-1_normal.jpg</profile_image_url>
    <url>http://www.destroytoday.com</url>
    <protected>false</protected>
    <followers_count>3330</followers_count>
    <profile_background_color>222222</profile_background_color>
    <profile_text_color>AAADB6</profile_text_color>
    <profile_link_color>00728F</profile_link_color>
    <profile_sidebar_fill_color>222222</profile_sidebar_fill_color>
    <profile_sidebar_border_color>222222</profile_sidebar_border_color>
    <friends_count>129</friends_count>
    <created_at>Tue May 20 00:08:47 +0000 2008</created_at>
    <favourites_count>122</favourites_count>
    <utc_offset>-18000</utc_offset>
    <time_zone>Eastern Time (US &amp; Canada)</time_zone>
    <profile_background_image_url>http://s.twimg.com/a/1257288876/images/themes/theme1/bg.png</profile_background_image_url>
    <profile_background_tile>false</profile_background_tile>
    <statuses_count>12159</statuses_count>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
    <following>false</following>
  </user>
</list>

In conclusion, I must say it’s pretty sad that an experienced Twitter developer needs to dig through the accepted developers’ work to be able to use the same features. I know Lists are new, but DestroyTwitter has been out far longer than the app I got this from. Also, the retweet and geo-location APIs have been public for months now, though the features aren’t completely implemented. DestroyTwitter doesn’t have millions of dollars worth of funding to bring to the table, but it has a pretty passionate developer who wants the latest features for his users.

[update] Thanks to Lim CHee Aun, who referred me to the draft documentation for Lists. It reveals a few more methods, including the creation and destruction of lists. The API URLs differ quite a bit from the Twitter API, using POST, DELETE, and GET with the same URL to create, delete, and return a list respectively. In the Twitter API, create and destroy are included in the URL itself. I’m hoping for the ability to bulk add/remove users to/from lists. That would seriously alleviate pressure from the hundreds of calls it would take otherwise.

DestroyTwitter is finally on Twitter. For the past nine months, I’ve used my personal Twitter account (@destroytoday) to post news and respond to feedback. That’s all fine by me, but I figured an account of its own would make things more official. Lots of people asked how I was able to get an account with the word “Twitter” in the name. In all honesty, I simply asked. Since DestroyTwitter has been around for a while (for a Twitter app), they were able to make an exception. If you haven’t done so already, go ahead and follow the @destroytwitter account—and keep an eye out over the next few weeks.

Christmas comes early as Twitter ups the API limit to 150 from 100. What does this mean?—more frequent canvas refreshes for users and a little breathing room for developers. I’m still holding out for unlimited API calls, but I know that will always be a dream.