Uploaded image for project: 'SnappyData'
  1. SnappyData
  2. SNAP-2436

Data mismatch in northWind queries during server node HA in column table test (data size used is 1 GB with redundancy set to 1)

    XMLWordPrintable

    Details

      Description

      Observed data mismatch in northWind Q26 after server node HA in column table test (data size used is 1 GB with redundancy set to 1).

      Q26:

      SELECT CompanyName FROM Customers WHERE CustomerID IN (SELECT CustomerID FROM Orders WHERE OrderDate BETWEEN Cast('1997-01-01' as TIMESTAMP) AND Cast('1997-12-31' as TIMESTAMP))

      Test Description:
      This test starts the snappy cluster and spark cluster.
      Test then runs the spark App for creating and loading data in column tables using northwind schema and data.
      It then executes the snappy job and sql script in parallel along with server HA and cluster restart.
      Snappy job executes and validate the northwind queries on the tables created and loaded through split mode.
      sql script only executes the northwind queries on the tables created and loaded through split mode.

      Exception Stack Trace from snappy job:

        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
      100  2235  100  2235    0     0  48818      0 --:--:-- --:--:-- --:--:-- 49666
      {
        "duration": "226.11 secs",
        "classPath": "io.snappydata.hydra.northwind.ValidateNWQueriesJob",
        "startTime": "2018-07-18T10:43:19.029-07:00",
        "context": "573787e2-io.snappydata.hydra.northwind.ValidateNWQueriesJob",
        "result": {
          "message": "java.lang.AssertionError: assertion failed: \nFor Q26 result count mismatch observed",
          "errorClass": "java.lang.RuntimeException",
          "stack": ["scala.Predef$.assert(Predef.scala:170)", "io.snappydata.hydra.SnappyTestUtils$.assertQueryFullResultSet(SnappyTestUtils.scala:136)", "io.snappydata.hydra.SnappyTestUtils$.assertJoinFullResultSet(SnappyTestUtils.scala:32)", "io.snappydata.hydra.northwind.NWTestUtil$$anonfun$validateSelectiveQueriesFullResultSet$1.apply(NWTestUtil.scala:475)", "io.snappydata.hydra.northwind.NWTestUtil$$anonfun$validateSelectiveQueriesFullResultSet$1.apply(NWTestUtil.scala:423)", "scala.collection.immutable.List.foreach(List.scala:381)", "io.snappydata.hydra.northwind.NWTestUtil$.validateSelectiveQueriesFullResultSet(NWTestUtil.scala:423)", "io.snappydata.hydra.northwind.ValidateNWQueriesJob$$anonfun$1.apply$mcV$sp(ValidateNWQueriesJob.scala:66)", "io.snappydata.hydra.northwind.ValidateNWQueriesJob$$anonfun$1.apply(ValidateNWQueriesJob.scala:42)", "io.snappydata.hydra.northwind.ValidateNWQueriesJob$$anonfun$1.apply(ValidateNWQueriesJob.scala:42)", "scala.util.Try$.apply(Try.scala:192)", "io.snappydata.hydra.northwind.ValidateNWQueriesJob.runSnappyJob(ValidateNWQueriesJob.scala:42)", "org.apache.spark.sql.SnappySQLJob$class.runJob(SnappySessionFactory.scala:122)", "io.snappydata.hydra.northwind.ValidateNWQueriesJob.runJob(ValidateNWQueriesJob.scala:28)", "spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:336)", "scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)", "scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)", "java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)", "java.lang.Thread.run(Thread.java:748)"]
        },
        "status": "ERROR",
        "jobId": "e4315202-e97e-461e-9381-951516694d94"
      }
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ssawant Swati Sawant
                Reporter:
                ssawant Swati Sawant
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: