故事是這樣開始的…

前一陣子, 團隊裡面的成員們, 正在為一段 story acceptance critira 的描述進行辯論, 後來得到兩個選項 :

A: Then the results page displayed
B: Then I should see results page

當大家正在針對上面這兩個選項進行辯論的過程中, 突然間有一個成員舉手了, 說 :

請問一下, 如果 scenario 用中文來寫, 會如何來描述???

這個問題真的敲醒我, 對ㄝ, 中文的話要怎麼表達這個句子.

A : 那麼結果頁面成功顯示
B : 那麼我能夠看見結果顯示成功

於是我開始反查 cucumber 裡面有沒有類似的作法, 結果發現 cucumber 在約兩個月 ( 10 月左右) 前提供了 i18n (多國語系) 的支援, 範例可以在這找到 :

../cucumber-2.4.0/examples/i18n/..

於是, 小弟我就寫一個 API 測試的範例來說明吧 :

Steps:

1. 在 feature 檔一開始定義繁體中文

#language: zh-TW

2. feature 檔寫法 :

過去我們習慣用 Feature: , 但是在 i18n 裡面我們可以用 功能: 表示.

功能: 
     為了避免愚蠢的錯誤
     作為一個 QA 初學者
     我希望有人能告訴我測試 API 結果

場景大綱: 代表 Scenario Outline:

場景大綱: 測試 API
 當我測試圖片API編號 <數值_1>
 那麼我驗證回傳編號 <驗證_1>

例子: 代表 Examples:

例子:
 | 數值_1 | 驗證_1 |
 | 1    | 1    |
 | 2    | 2    |
 | 20   | 10   |

當大功告成之後, 我們就來實作 step 檔吧 :

When /我測試圖片API編號 (\d+)/ do |op|
    @response = @req.get("photos/#{op}")
end

Then /我驗證回傳編號 (\w+)/ do |op|
    json = Json_Parser.new(@response)
    ids = json.get_json_values_by_key("id")
    if not ids.eql? "#{op}".to_i
        raise "expect id : #{op}, actual id : #{ids}"
    end
end

在這裡我是引用 rest_client 及 json, 這兩個 ruby 的 package 來處理 API 的 request 及 response, 詳細的實作在這裡就不多做說明了.

當一切都完成後, 我們就來執行 cucucmber 吧!

cucumber

結果 :

經過這一番努力後, 得到一個結論 : 

誰說中文不能寫測試案例 :)

Github : 

https://github.com/jerseysu/ruby-cucumber-rest-api

[Cucumber] 如何用 rakefile 重跑 Cucumber Fail 的 Features ???

Use --format rerun --out rerun.txt to write out failing features. You can rerun them with cucumber @rerun.txt. 

• :run 用來執行任務
• :rerun 用來重跑任務

require 'rubygems'
require 'cucumber'
require 'cucumber/rake/task'

namespace :test do

    desc 'ExecuteTestCase'
    task :execute do
        Cucumber::Rake::Task.new(:run) do |t|
            t.cucumber_opts = "-f pretty --format rerun --out rerun.txt"
        end

        is_first_run_successful = run_rake_task ("run")

        if not is_first_run_successful
            Cucumber::Rake::Task.new(:rerun) do |t|
                t.cucumber_opts = "@rerun.txt"
            end

            run_rake_task("rerun")
        end
    end

    def run_rake_task(name)
        begin
            Rake::Task[name].invoke
        rescue Exception => e
            puts e
            return false
        end
        true
    end
end

$ rake test:execute

How to add android system log into calabash-android test scenarios

Add follow adb command into app_life_cycle_hook.rb file, and the system log will log into file for each scenarios

Before do |scenario|
  `#{adb_command} logcat -c` # clears logcat output
  #Start Test Server
  start_test_server_in_background
end

After do |scenario|
  `#{adb_command} logcat -v time 2>&1 -d > $REPORT_PATH/sys_log.txt` # dumps log in time format to file and exits
  embed('sys_log.txt', 'text/plain', 'sys_log.txt')
  if scenario.failed?
    screenshot_embed
  end
  shutdown_test_server
end

1. BDD不是工具，而是一套流程和一系列实践。它需要团队成员的通力合作，可以帮助整个团队更好的理解业务，理解软件。
2. Cucumber作为支持BDD的一种工具，不单单是自动化测试工具。在解决了Cucumber的一些问题后，团队可以更加有效的使用。
3. Cucumber，Concordion和Guage各有不同，选择一款适合团队自身需要的工具，也能保证团队顺利运作，少走弯路。

• Feature file
• Step Definitions file
• Configuration

Scenario: User logs in

  Given I am on the homepage

  When I click on the "Login" button

   And I fill in the "Email" field with "me@example.com"

   And I fill in the "Password" field with "secret"

   And I click on "Submit"

  Then I should see "Welcome to the app, John Doe"

Scenario: User logs in

  Given I am on the homepage

  When I log in

  Then I should see a login notification

Step Definitions	Keep the step definitions unique. If your step definition names aren’t unique, then you will find during runtime the following exception is thrown - ambiguous match.
Hard-coding	As you know you shouldn't be hard coding parameters into your code. This applies to site URL’s as well to allow flexibility to globally change the value.
Reusable Objects	At the beginning of any automation project, you need to think about scalability and maintainability. A good programming practice is Don’t Repeat Yourself (or DRY) by creating reusable code. You will find many different ways to share code between Cucumber scenarios. Be careful! Call other step definitions. Call step definitions from other step definitions by calling steps helper. (Avoid) Use many_steps helper. It does everything that steps helper does, but gives you meaningful stack traces in case something goes wrong. (Avoid) Use a Page Object Pattern should be mapped to a Ruby class file, and each method within the class represents a page object on the page. (Better Choice) Why use the Page Object Pattern? Readable DSL Promotes reusable code Centralizes UI coupling — one place to make changes rather than multiple places throughout step definitions
Code	No sleep statements. It is better to use method or function with a built-in timeout.
RSpec Expectations	The rspec-expectations library is bundled with many built-in matchers for testing validation. Each of the matchers can be used with expect(..).to or expect(..).not_to to define positive and negative expectations respectively on an object under test. I recommend using rspec-expectations matchers over Selenium matchers for Ruby testing frameworks.

- Cucumber 的組態設定 

在 Ruby 使用 Cucumber 是一件很簡單的事情, 需包含下列這些組態檔案:

cucumber.yml	It simplifies command-line execution by defining reusable profiles within the cucumber.yml file.
Gemfile	The Gemfile is used to manage dependencies for a Ruby project. We should also pin the versions of these Ruby gems. If you don’t pin a Ruby version, it will auto update and break your automation.
Rakefile	Rakefile is a software task management and build automation tool. It allows you to specify tasks and describe dependencies as well as group tasks in namespaces.
support/env.rb	The support/env.rb is an environment configuration file for Cucumber. This file is designed to provide flexibility by allowing you to do things easily, like toggle your execution environment from local to remote. To fully understand all the capabilities of this support file, you should take more time to read up on all of the capabilities.

心得

蠻喜歡文章最後提到的一段話

BDD (Cucumber) allows for team collaboration early in the planning and development phases. Keep the numbers down. Ask yourself and the team: “If I can write a low-level test for this user story, should I write an UI acceptance test?” We should automate only the things that need to be automated at the UI level. It is a known fact that low-level tests are more reliable compared to UI tests. Cucumber is code, and all types of programming require thoughts upfront before writing automated tests. It is so important to lay a test automation foundation that embraces coding standards and follows design patterns that everyone will follow during automation development.

其實 BDD 最大的精神就是協同合作 (collaboration), 而裡面有一個問題 “If I can write a low-level test for this user story, should I write an UI acceptance test?”, 

自己蠻認同他的答案, 也就是說我們應該只自動化在 UI 層級那些需要被自動化的部分, 因為這是一大家都知道的事實.

與其說 Cucumber 是 code 的一部份, 還不如說每當我們在寫自動化測試之前, 先制定 coding standard 和 遵循的設計模式 的方法, 讓這些方法成為自動化測試的基礎, 因此可以讓更多人投入來寫自動化測試.

[Reference]

1. Write Great Cucumber Test