1. BDD不是工具，而是一套流程和一系列实践。它需要团队成员的通力合作，可以帮助整个团队更好的理解业务，理解软件。
2. Cucumber作为支持BDD的一种工具，不单单是自动化测试工具。在解决了Cucumber的一些问题后，团队可以更加有效的使用。
3. Cucumber，Concordion和Guage各有不同，选择一款适合团队自身需要的工具，也能保证团队顺利运作，少走弯路。

從 QA 來看工程生產力

這幾天在 Google Testing Blog 上面看到一篇文章在探討從 QA 到工程的生產力, From QA to Engineering Productivity 頗有趣的, 只好來寫個心得.

在 Google 創建初期時, 是由一小群工程師組成, 其中包含建置, 測試和出版. 但隨者用戶的增加及產品成長, 工程師們開始專注於下面的角色:

• Test Engineers (TEs) – tested new products and systems integration
• Release Engineers (REs) – pushed bits into production
• Site Reliability Engineers (SREs) – managed systems and data centers 24x7.

文章中除了探討軟體品質的改善及這些角色在 Google 裡主要的職責, 也包含許多珍貴及重要的測試轉型經驗, 但關於 REs 跟 SREs 的分享, 會在往後的文章中再來探討.

在初期的 Google團隊中, 其實蠻重度依賴手動執行. 因此團隊企圖導入自動化測試, 在產品還沒有很多整合及團隊還很小的時候是有效用的. 然而, 當 Google 開始成長, 冗長的手動測試週期卻會使團隊陷入是一個停滯不前的迴圈並且拖慢 features 的發行. 此外, 由於在開發的後期才發現 Bugs, 使得團隊花了更長的時間在修復問題. 所以文章中判斷, 若是在開發的初期投入自動化測試, 將有助於加速解決上列的問題.

However, as Google grew, longer and longer manual test cycles bogged down iterations and delayed feature launches. Also, since we identified bugs later in the development cycle, it took us longer and longer to fix them. We determined that pushing testing upstream via automation would help address these issues and accelerate velocity.

當 Google 將手動測試轉型成為自動化流程的同時, 也出現了兩種測試的角色:

• Test Engineers (TEs) – With their deep product knowledge and test/quality domain expertise, TEs focused on what should be tested.
• Software Engineers in Test (SETs) – Originally software engineers with deep infrastructure and tooling expertise, SETs built the frameworks and packages required to implement automation.

而這樣的轉變也帶來明顯的影響:

• 自動化使測試變得更加有效率及正確性 (deterministic) (例如: 改善執行時間, 消除 sources of flakiness 等)
• Matrics driven 工程增加 (例如: 帶出高品質的程式碼及功能覆蓋率)
• 手動的測試減少到只做新功能的手動驗證，通常只做 End to End Testing, 和跨產品的整合測試.
• 測試工程師 (TEs) 發展成為產品知識的專家 (developed extreme depth of knowledge). 同時也是 Product Team 自動化測試和整合測試的知識專家 (for product teams that needed expertise in test automation and integration).

因此,  TEs (測試工程師) 負責的範圍將漸漸轉型為：

• 編寫腳本自動化測試
• 開發工具讓開發人員可以測試自己的 code
• 不斷地設計出更好及更具有創造性的方式來找出產品的的弱點

SETs (與測試工程師及其他工程師協同合作) 建立跨產品的自動化測試工具包, 使得產品可以快速的發行.

SETs 在初期注重於建立減少測試週期的工具, 這是為了減少從 product code 到發行這個費時及耗費人力的階段. 在 Google, SETs 也做出許多在軟體測試領域有名的工具, 如 webdriver improvements, protractor, espresso, EarlGrey, martian proxy, karma, 和 GoogleTest.

SETs 也樂於與其他測試工程師分享, 並且建立社群. 業界也開始擁抱這樣的測試科學 (Test Engineering discipline), 有些軟體公司也將軟體工程師轉型成為這樣的角色, 並且發表文章, 更推行測試先行 (Test-Driven Development) 成為業界的主流.

SETs were interested in sharing and collaborating with others in the industry and established conferences. The industry has also embraced the Test Engineering discipline, as other companies hired software engineers into similar roles, published articles, and drove Test-Driven Development into mainstream practices.

透過這些努力, 測試的週期時間有明顯的下降, 有趣的是, 整體的開發速度並沒有相應地提升，反而其他階段卻成為開發週期的瓶頸. 因此, SETs 開始開發下列工具以加速產品的開發週期:

• Extends IDE 使 Write Code 及 Code Review 更加容易, 進而縮短 “Write Code” 週期
• 自動化 release 審核，縮短 “Release Code” 週期
• 自動化即時 (real time) 產生 system log 驗證及異常檢測，協助自動化 production monitoring.
• 自動化開發人員的生產力的測量，有助於了解 what’s working and what isn’t.

總結來說, SETs 自然進展成從只支援產品測試的 efforts 到支援產品開發的 efforts. 這個角色現在也成為一個涵蓋更廣泛工程生產力的議程. (Their role now encompassed a much broader Engineering Productivity agenda)

鑑於擴展 SET 的 Charter, Google 也想要有一個能夠反映這個職位的職稱. 有趣的是, 要怎麼給這個職稱呢？ 作者說他們授權給 SETs 選擇新的職稱, 而他們絕大多數（91％）選擇了軟體工程師 (Software Engineer) , 工具 (Tools) 和基礎設施(Infrastruture) (簡稱SETI) 這樣的稱位. 今天, SETIs 和 TEs 還是非常密切的合作在提升整體的開發週期, 來消除從 feature 到 產品的階段所有的摩擦. (with a goal of eliminating all friction from getting features into production)

其實從 2010 到現在, 自己從 Desktop, Web frontend, Backend API, 到 Mobile 做 Testing. 也約 5 年的時間在軟體測試領域發展, 看到 Google 的文章, 覺得相當興奮, 真希望有一天能夠加入這樣的開發團隊. 雖然台灣還沒有特別重視這樣的職位, 但相信在這個講求 Aglie, Continually Delivery 和品質的時代, 這一天是不遠了… （有夠樂觀 )

[Reference]
Google Test Blog : From qa to engineering productivity

• Feature file
• Step Definitions file
• Configuration

Scenario: User logs in

  Given I am on the homepage

  When I click on the "Login" button

   And I fill in the "Email" field with "me@example.com"

   And I fill in the "Password" field with "secret"

   And I click on "Submit"

  Then I should see "Welcome to the app, John Doe"

Scenario: User logs in

  Given I am on the homepage

  When I log in

  Then I should see a login notification

Step Definitions	Keep the step definitions unique. If your step definition names aren’t unique, then you will find during runtime the following exception is thrown - ambiguous match.
Hard-coding	As you know you shouldn't be hard coding parameters into your code. This applies to site URL’s as well to allow flexibility to globally change the value.
Reusable Objects	At the beginning of any automation project, you need to think about scalability and maintainability. A good programming practice is Don’t Repeat Yourself (or DRY) by creating reusable code. You will find many different ways to share code between Cucumber scenarios. Be careful! Call other step definitions. Call step definitions from other step definitions by calling steps helper. (Avoid) Use many_steps helper. It does everything that steps helper does, but gives you meaningful stack traces in case something goes wrong. (Avoid) Use a Page Object Pattern should be mapped to a Ruby class file, and each method within the class represents a page object on the page. (Better Choice) Why use the Page Object Pattern? Readable DSL Promotes reusable code Centralizes UI coupling — one place to make changes rather than multiple places throughout step definitions
Code	No sleep statements. It is better to use method or function with a built-in timeout.
RSpec Expectations	The rspec-expectations library is bundled with many built-in matchers for testing validation. Each of the matchers can be used with expect(..).to or expect(..).not_to to define positive and negative expectations respectively on an object under test. I recommend using rspec-expectations matchers over Selenium matchers for Ruby testing frameworks.

- Cucumber 的組態設定 

在 Ruby 使用 Cucumber 是一件很簡單的事情, 需包含下列這些組態檔案:

cucumber.yml	It simplifies command-line execution by defining reusable profiles within the cucumber.yml file.
Gemfile	The Gemfile is used to manage dependencies for a Ruby project. We should also pin the versions of these Ruby gems. If you don’t pin a Ruby version, it will auto update and break your automation.
Rakefile	Rakefile is a software task management and build automation tool. It allows you to specify tasks and describe dependencies as well as group tasks in namespaces.
support/env.rb	The support/env.rb is an environment configuration file for Cucumber. This file is designed to provide flexibility by allowing you to do things easily, like toggle your execution environment from local to remote. To fully understand all the capabilities of this support file, you should take more time to read up on all of the capabilities.

心得

蠻喜歡文章最後提到的一段話

BDD (Cucumber) allows for team collaboration early in the planning and development phases. Keep the numbers down. Ask yourself and the team: “If I can write a low-level test for this user story, should I write an UI acceptance test?” We should automate only the things that need to be automated at the UI level. It is a known fact that low-level tests are more reliable compared to UI tests. Cucumber is code, and all types of programming require thoughts upfront before writing automated tests. It is so important to lay a test automation foundation that embraces coding standards and follows design patterns that everyone will follow during automation development.

其實 BDD 最大的精神就是協同合作 (collaboration), 而裡面有一個問題 “If I can write a low-level test for this user story, should I write an UI acceptance test?”, 

自己蠻認同他的答案, 也就是說我們應該只自動化在 UI 層級那些需要被自動化的部分, 因為這是一大家都知道的事實.

與其說 Cucumber 是 code 的一部份, 還不如說每當我們在寫自動化測試之前, 先制定 coding standard 和 遵循的設計模式 的方法, 讓這些方法成為自動化測試的基礎, 因此可以讓更多人投入來寫自動化測試.

[Reference]

1. Write Great Cucumber Test

1. Use rebase to edit previous 3 commit

git rebase -i HEAD~3

2. Open editor to edit the commit which you want to change

pick f37e150 xxxx
pick f15f439 xxxx
edit 647aab8 xxxx

3. amend to your commit

git commit --amend

4. After change your commit then continue rebase

git rebase --continue

5. Done.

git config --global core.editor /usr/bin/vi

1. Git-工具-重寫歷史
2. Change vi on mac os

STEP 1: Export Posts from Tumblr Using Tumblr2Wordpress

STEP 2: Convert Wordpress backup file into a Blogger XML file

STEP 3: Import into Blogger

Calabash-iOS on Physical Devices

0. Prepare the environment

1. Link calabash.framework to your XCode project

$ bundle exec calabash-ios download

After that, you have build your apps on real machine.

2. Install and generate Calabash-ios folder

$ gem install calabash-cucumber
$ calabash-ios gen

3. Start to run your test

• BUNDLE_ID : Your apps bundle_ID
• DEVICE_TARGET : Your devices UDID
• DEVICE_ENDPOINT : Your devices IP

$ export BUNDLE_ID=com.example.MyApp-cal
$ export DEVICE_TARGET=< UDID >
$ export DEVICE_ENDPOINT=http://<ip>:37265
$ bundle exec cucumber

Tips: To get UDID from command line

udid=$(ideviceinfo | grep UniqueDeviceID | awk '{ print $2}')

[Reference]

1. Set calabash-ios framework
https://github.com/calabash/calabash-ios/wiki/Tutorial%3A–Creating-a-cal-Target

2. Name your own devices
https://github.com/calabash/calabash-ios/wiki/Improving-Network-Stability#tip-name-your-devices

3. Test on real devices
https://github.com/calabash/calabash-ios/wiki/Testing-on-Physical-Devices

[Android] 

adb devices

gem install calabash-android

gem install cucumber

calabash-android gen

calabash-android setup

- keystore location: ~/.android/debug.keystore 
- password: android 
- alias: androiddebugkey

bundle exec calabash-android run <apk>

ADB_DEVICE_ARG='$DEVICE_ID' calabash-android run <apk>