Thanks for the feedback and encouragement so far. This project has been fun and I hope you keep enjoying it so far.
Refactoring Required
I knew going in that the first cut of this application needed some serious refactoring. The project had dependencies on internet connections , numerous classes contained redundant code and the user interface was lacking. In this first set of refactorings I hope to demonstrate how you can design your applications to be more testable and how to make the code cleaner at the same time. I will also attempt to make the UI better with my feeble UI skills.
First Refactoring: Extracting Interfaces
The first cut of the Twitter Conversations application took a dependency on being connected to the Internet for testing. In order to remove this dependency I decided to extract the an interface from the Twitter.Communications.TwitterRequest class. This process is simple when using a tool like ReSharper. I simply opened the class source code file, Right Clicked, Selected Refactor->Extract Interface from the popup menu. The following screen shows the Extract Interface dialog from ReSharper.
This process yielded the following interface
1: Public interface ITwitterRequest
2: Function GetTwitterRequest(ByVal URL As String) As String
3: end interface
After refactoring the interface from the Twitter.Communication.TwitterRequest class I created a new Twitter.Interfaces project. This project would all interfaces for this project including another new interface ITwitterCredentials. I created the ITwitterCredentials interface in order to provide a slightly cleaner design for creating instances of the TwitterRequest class.
1: Public Interface ITwitterCredential
2: Property UserName() As String
3: Property Password() As String
4: End Interface
Once I created the ITwitterCredentials interface I implemented it on a new class Twitter.Communication.TwitterCredentials. I also needed to refactor the guts of the TwitterRequest class. The new code for this class now looks like:
1: Public Class TwitterRequest
2: Implements Twitter.Interfaces.ITwitterRequest
3: Private _Credentials As Twitter.Interfaces.ITwitterCredential
4:
5: Sub New(ByVal Credentials As Twitter.Interfaces.ITwitterCredential)
6: Me._Credentials = Credentials
7: End Sub
8:
9: Public Function GetTwitterRequest(ByVal URL As String) As String Implements Twitter.Interfaces.ITwitterRequest.GetTwitterRequest
10: Dim Credentials As New NetworkCredential(Me._Credentials.UserName, Me._Credentials.Password)
11:
12: Dim Request As HttpWebRequest = HttpWebRequest.Create(URL)
13: Request.Method = "POST"
14: Request.Credentials = Credentials
15:
16: Dim Response As WebResponse = Request.GetResponse
17: Dim Reader As New StreamReader(Response.GetResponseStream)
18: Dim Results As String = Reader.ReadToEnd
19: Return Results
20:
21: End Function
22: End Class
After extracting these interfaces I changed the constructors for Twitter.API.User and Twitter.Message.API to receive an ITwitterRequest rather than a concrete class.The following code demonstrates the constructor for the Twitter.API.User class.
1: Public Class User
2:
3: Dim Communicator As Twitter.Interfaces.ITwitterRequest = Nothing
4: Dim JsonSerializer As New System.Web.Script.Serialization.JavaScriptSerializer
5:
6: Sub New(ByVal TwitterCommunicator As Twitter.Interfaces.ITwitterRequest)
7: Me.Communicator = TwitterCommunicator
8: End Sub
Of course at this point all of our tests were broken but I waited to do a little more refactoring before fixing the tests.
Second Refactoring: Creating the Disconnected Twitter Service
The next step in the process was to create a disconnected version of Twitter. To accomplish this I took the following steps:
- Extracted JSON strings from the various calls to Twitter and put them into our testing project as embedded resource files.
- Added code to the testing project that could be used to extract the embedded text files
- Implemented the ITwitterRequest interface on a new class called TwitterRequestMock.
- Implemented the function GetTwitterRequest to return the appropriate text resource based on the URL.
The first item on the list was to extract the JSON data from actual calls to Twitter. To satisfy the unit tests I created three file extracts:
- singleuser.txt - Contains the information for my user account
- friends.txt – A list of people I follow
- messages.txt – A list of twitter updates
All three of these files were marked as Embedded Resource files in the Visual Studio project.
The next step was to add code that would allow these files to be extracted from the testing assembly. the following code extracts resources from the currently running assembly (i.e. our unit tests)
1: Imports System.IO
2: Imports System.Reflection
3:
4: Public Class ObjectMother
5: Public Shared Function GetEmbeddedResource(ByVal ResourceName As String) As String
6: Dim ResStream As Stream = System.Reflection.Assembly.GetExecutingAssembly.GetManifestResourceStream(ResourceName)
7: Dim oStreamReader As New StreamReader(ResStream)
8: Dim RetVal As String = oStreamReader.ReadToEnd
9: Return RetVal
10: End Function
11: End Class
Finally a new class TwitterRequestMock was created. The purpose of this class was to return the proper embedded resource files based on the URL passed to a request. The following code demonstrates the implementation of this:
1: Public Class TwitterRequestMock
2: Implements Twitter.Interfaces.ITwitterRequest
3:
4: Private _Credentials As Twitter.Interfaces.ITwitterCredential
5:
6: Sub New(ByVal Credentials As Twitter.Interfaces.ITwitterCredential)
7: Me._Credentials = Credentials
8: End Sub
9:
10: Public Function GetTwitterRequest(ByVal URL As String) As String Implements Interfaces.ITwitterRequest.GetTwitterRequest
11: Dim ReturnValue As String = ""
12:
13: '-- urls we use so far
14: 'Const UserMessageURL As String = "http://twitter.com/statuses/user_timeline/<<USERID>>.json"
15: 'Const MyMessagesURL As String = "http://twitter.com/statuses/friends_timeline.json?count=200"
16: 'Const FriendsURL As String = "http://twitter.com/statuses/friends.json"
17: 'Const GetUserURL As String = "http://twitter.com/users/show/<<USERID>>.json"
18: 'Const CredentialURL As String = "http://twitter.com/account/verify_credentials.json"
19:
20: If URL.Contains("verify_credentials") OrElse URL.Contains("users/show/") Then 21: ReturnValue = ObjectMother.GetEmbeddedResource("Twitter.UnitTests.singleuser.txt") 22: ElseIf URL.Contains("friends_timeline") OrElse URL.Contains("user_timeline") Then 23: ReturnValue = ObjectMother.GetEmbeddedResource("Twitter.UnitTests.messages.txt") 24: ElseIf URL.Contains("friends.json") Then 25: ReturnValue = ObjectMother.GetEmbeddedResource("Twitter.UnitTests.friends.txt") 26: End If
27:
28: Return ReturnValue
29: End Function
30: End Class
I discussed this method with Jeremy Miller and his comment was if possible the text for the JSON strings should be embedded into the unit test if at all possible. I didn’t follow this recommendation because the JSON returns strings are long and very ugly. What are your thoughts on this ?
Third Refactoring: Fixing the Unit Tests
Once this class was implemented I went in and fixed up the unit tests. This was actually pretty simple. Basically I refactored the constructor and <Setup()> of the UnitTests class.
1: <TestFixture()> _
2: Public Class UnitTests
3:
4: Dim UserAPI As Twitter.API.User = Nothing
5: Dim MessageAPI As Twitter.API.Message = Nothing
6: Dim Communicator As Twitter.Interfaces.ITwitterRequest = Nothing
7:
8: <SetUp()> _
9: Sub Setup()
10: Me.Communicator = New TwitterRequestMock(New Twitter.Communication.TwitterCredentials)
11:
12: Me.UserAPI = New Twitter.API.User(Me.Communicator)
13: Me.MessageAPI = New Twitter.API.Message(Me.Communicator)
14: End Sub
Now the tests for this application can be run in a disconnected mode.
Fourth Refactoring: Removing Redundant Code
The next step in this refactoring process was to clean up redundant code from the Twitter.API.User and Twitter.API.Message classes. In the first version each method was responsible for parsing data and creating collections of Messages and Users. The code started like the following snippet:
1: Function GetUserMessages(ByVal UserID As String) As Twitter.Domain.Message()
2: Dim ReturnMessages As New List(Of Twitter.Domain.Message)
3:
4: Dim UserObjects As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(UserMessageURL.Replace("<<USERID>>", UserID))) 5:
6: For Each UserObject As Object In CType(UserObjects, Array)
7: ReturnMessages.Add(New Twitter.Domain.Message With { _ 8: .MessageID = UserObject("id"), _ 9: .MessageDate = UserObject("created_at"), _ 10: .MessageContent = UserObject("text"), _ 11: .UserID = UserObject("user")("id"), _ 12: .UserName = UserObject("user")("name"), _ 13: .ImageURL = UserObject("user")("profile_image_url")}) 14: Next
15: Return ReturnMessages.ToArray
16:
17: End Function
The redundant code is the code contained in the For loop. I decided to extract this code into its own method.
1: Sub PopulateMessages(ByVal ListToPopulate As List(Of Twitter.Domain.Message), ByVal Contents As Object)
2: For Each UserObject As Object In CType(Contents, Array)
3: ListToPopulate.Add(New Twitter.Domain.Message With { _ 4: .MessageID = UserObject("id"), _ 5: .MessageDate = UserObject("created_at"), _ 6: .MessageContent = UserObject("text"), _ 7: .UserID = UserObject("user")("id"), _ 8: .UserName = UserObject("user")("name"), _ 9: .ImageURL = UserObject("user")("profile_image_url")}) 10:
11: Next
12: End Sub
Now the code for parsing off message looks like this:
1: Function GetUserMessages(ByVal UserID As String) As Twitter.Domain.Message()
2: Dim ReturnMessages As New List(Of Twitter.Domain.Message)
3: Dim UserObjects As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(UserMessageURL.Replace("<<USERID>>", UserID))) 4: Me.PopulateMessages(ReturnMessages, UserObjects)
5: Return ReturnMessages.ToArray
6: End Function
The Twitter.API.User Class is had a slightly different refactoring. Because this API returns a lot of single user information it was necessary to create a function to return a single user object. When a method returned multiople user records a looping structure similar to the one for messages was used.
1: Public Function GetUsers() As Twitter.Domain.User()
2: Dim Users As New List(Of Twitter.Domain.User)
3: Dim UserObjects As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(FriendsURL))
4: Me.PopulateUsers(Users, UserObjects)
5: Return Users.ToArray
6: End Function
7:
8: Public Function GetUser(ByVal id As String) As Twitter.Domain.User
9: Dim Users As New List(Of Twitter.Domain.User)
10: Dim UserObject As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(GetUserURL.Replace("<<USERID>>", id.ToString))) 11: Return Me.GetUserFromObject(UserObject)
12: End Function
13: Public Function GetMyUser() As Twitter.Domain.User
14: Dim UserObject As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(CredentialURL))
15: Return Me.GetUserFromObject(UserObject)
16: End Function
17: Function GetUserFromObject(ByVal UserObject As Object) As Twitter.Domain.User
18: Return New Twitter.Domain.User With { _ 19: .UserID = UserObject("id"), _ 20: .UserName = UserObject("name"), _ 21: .ScreenName = UserObject("screen_name"), _ 22: .ImageURL = UserObject("profile_image_url"), _ 23: .Followers = UserObject("followers_count")} 24: End Function
25: Sub PopulateUsers(ByVal ListToPopulate As List(Of Twitter.Domain.User), ByVal Contents As Object)
26: For Each UserObject As Object In CType(Contents, Array)
27: ListToPopulate.Add(Me.GetUserFromObject(UserObject))
28: Next
29: End Sub
The nice thing about these refactoring is that when a new property is added to a domain object you only need to change the code in one place.
Fourth Refactoring: The WPF Interface
The final refactoring I added was to implement a slightly improved interface. This basically consisted of changing the code that calls the Twitter API’s we write and adding three user interface elements. The elements added to the screen consisted of two fields for capturing username and password respectively and a new display element for the showing the Twitter users image. The VB code and XAML code now looks like this:
1: Private Sub cmdGetThread_Click(ByVal sender As Object, ByVal e As System.Windows.RoutedEventArgs) Handles cmdGetThread.Click
2: Dim Communicator As Twitter.Interfaces.ITwitterRequest = New Twitter.Communication.TwitterRequest(New Twitter.Communication.TwitterCredentials With {.UserName = Me.txtUserName.Text, .Password = Me.txtPassword.Password}) 3: Me.lstResults.ItemsSource = Me.LoadConversation(New Twitter.API.Message(Communicator), New Twitter.API.User(Communicator), Me.txtCriteria.Text)
4: End Sub
5:
6:
7: Function LoadConversation(ByVal MessageAPI As Twitter.API.Message, ByVal UserAPI As Twitter.API.User, ByVal TextCriteria As String) As Twitter.Domain.Message()
8:
9: '-- create list of users from comma (,) delimited list of names in text box
10: '-- TODO we should scrub this
11: Dim UserQuery As New List(Of Twitter.Domain.User)
12: For Each UserName As String In TextCriteria.Split(",") 13: UserQuery.Add(UserAPI.GetUser(UserName))
14: Next
15:
16: '-- get messages for these users
17: Dim messages As Twitter.Domain.Message() = MessageAPI.GetMultipleUserMessages(UserQuery.ToArray())
18:
19: '-- filter messages based on who is in the contents
20: Dim FilteredMessages = _
21: From Message _
22: In messages _
23: Where MatchesCriteria(Message.MessageContent, Me.txtCriteria.Text.Split(",")) _ 24: Order By Message.MessageDate Descending
25:
26: Return FilteredMessages.ToArray
27:
28:
29: End Function
1: <Grid x:Name="LayoutRoot">
2: <StackPanel >
3: <StackPanel Width="Auto" Orientation="Horizontal">
4: <TextBlock Text="User Name" Height="22" Width="80"/>
5: <TextBox Text="" TextWrapping="Wrap" x:Name="txtUserName" Width="151" Height="27"/>
6: <TextBlock Text="Password" Height="22" Width="67.957"/>
7: <PasswordBox x:Name="txtPassword" Width="143" Height="27"/>
8: </StackPanel>
9:
10: <StackPanel Width="Auto" Orientation="Horizontal">
11: <TextBlock Text="Screen Names" Height="22" Width="80"/>
12: <TextBox Text="rodpaddock,bellware" TextWrapping="Wrap" x:Name="txtCriteria" Width="500" Height="22"/>
13: <Button Content="Get Thread" x:Name="cmdGetThread"/>
14: </StackPanel>
15: <StackPanel>
16: <ScrollViewer Width="Auto" Height="600">
17: <ListBox Width="Auto" Height="600" IsSynchronizedWithCurrentItem="True" x:Name="lstResults" >
18: <ListBox.ItemTemplate>
19: <DataTemplate>
20: <StackPanel Orientation="Horizontal" >
21: <Image Height="100" Width="100" Source="{Binding Path=ImageURL}"/> 22: <StackPanel>
23: <TextBlock Text="{Binding Path=MessageContent}"/> 24: <TextBlock Text="{Binding Path=MessageDate}"/> 25: <TextBlock Text="{Binding Path=UserName}"/> 26: <TextBlock Text="----------"/>
27: </StackPanel>
28:
29: </StackPanel>
30:
31: </DataTemplate>
32: </ListBox.ItemTemplate>
33:
34: </ListBox>
35: </ScrollViewer>
36: </StackPanel>
37: </StackPanel>
38: </Grid>
One item to notice is that all of the code for calling the Twitter API is no longer embedded in the Click() event of the Search button. It has been extracted into its own method and no longer relies on user interface elements. This refactoring was done with integration testing in mind. In a later installment we will be looking at testing the user interface along with just the API calls.
Now the user interface looks like:
Google Code
The source code for this project can now be found on Google Code The URL for this project is:
http://twitterplayground.googlecode.com/svn/trunk/twitterplayground-read-only
More Coming
I hope you have enjoyed these posts so far. Lets take a look at the list from the last post and strike out some tasks that were accomplished in this post:
1. Introduce a Dependency Injection container. For this application I want to use StructureMap.
2. Introduce more mocking to remove dependencies on Twitter (maybe we can write a Twitter mock layer)
3. Further refine the design of the libraries. There’s still redundant code left we can remove.
4. Improve the UI and create some alternate UI’s with MVC or maybe Silverlight
5. Open up the code to contributions from other developers.
We accomplished quite a bit in this post. Is all of the code done? Not by any stretch. I plan on revisiting all of the items on the list in every refactoring (rinse-lather-repeat)… I will also add new items to the list, like:
- Adding Proxy code to the communications class.
- Adding error handling to the communications class.
- Creating some mechanism for preserving credentials and other settings
RE #5: I did put it in Google code and would be happy to have other contributors. I would especially appreciate help with the UI.
NOTE: One of the cool things about this process so far is the confidence I have in changes I make to the code. Having a repeatable set of tests is invaluable.
Thanks
Rodman
