Saturday, 6 May 2023

AWS Short Notes(In Progress)

AWS CloudFormation:

It is a template used to CRUD infrastructure in AWS environment. It has template format which is described below.

AWSTemplateFormatVersion: "version date"  (section (optional) identifies the capabilities of the template. The latest template format version is 2010-09-09 and is currently the only valid value.)

         JSON way to represent:

        "AWSTemplateFormatVersion" : "2010-09-09"

        YAML way to represent:

        AWSTemplateFormatVersion: "2010-09-09"

Description:  The Description section (optional) enables you to include comments about your template. [ 0 and 1024 bytes in length]

eg:    JSON way

           "Description" : "Here are some details about the template."

       YAML way

            Description: >

                   Here are some details about the template.


Metadata:

Parameters:

Mappings:

Conditions:

Transform:

Resources: (mandatory field)

Outputs:

Take Away: 

1)Resources is mandatory field.

2)If we have both Description and AWSTemplateFormatVersion then its mandatory for the description field to follow AWSTemplateFormatVersion . Important restriction 




Friday, 5 May 2023

AWS CLI commands [In progress]

1)aws configure

2)aws configure list : will list config being used to connect to aws services. Credentials are present inside shared credentials file located in .aws/credentials file and other configs like region is present inside .aws/config



Thursday, 23 March 2023

Pytest usage in Pycharm

Setup Pytest in Pycharm for running test cases 

 1)Install pytest package from python interpreter option by navigating to preferences ---->python                       interpreter

2)Create new run/debug configuration for test cases wherein following information need to be given:

    2.1) Name of the run/debug give it as per your choice.

    2.2)  Target: This will the folder under which you want to run tests . It has different options like script,              custom etc.

          * Use script if want to execute all test cases under folder/project.

         * Use custom if want to make dynamic file selection like I did by passing --tb=1 as argument . This                will   pick current file and run test cases inside that file.

3)Working directory: This is last and mandatory to specify where to pick files from. 

4)Last change is make sure you are using pytest as test runner and it is selected in preferences-->Python Integrated Tools---->Testing--->Default test Runner --(pytest)


Below is step 2 configuration screenshot



  


Thursday, 5 January 2023

Mongo Commands

Basic commands in Mongo:

1)use dbname: is used to select specific database that need to be used. If no database is present , it will create new db.
2)show collections : is used to list all collection in db.
3)db.collectionName.find({}). : is used to list all documents in specific collection.
4)db.collectionName.countDocuments({}) ::  returns number of documents present in collection.
5)db.collectionName.find().sort({"date":-1}) :: sort by date in descending order .
6)db.collectionName.find().sort({"date":1}) :: sort by date in ascending order.
7)db.collectionName.deleteMany({}) :: delete all documents in collection
8)db.collectionName.deleteOne({}) :: 
9) db. collectionName .insertMany():  will create collection if does not exist with multiple insert.
10)aggregation examples: we can use aggregation whenever we want to aggregate data. Mongo supports aggregation using pipeline wherein pipeline consists of different stages. Each stage can do different task like filtering, matching , sorting on the selected set of documents. I feel like aggregation is important so putting some examples for reference. 


10.1) created demo collection with multiple records.

db.demo.insertMany([{
"tid": 1,
"sq": 1,
"state": "assigned",
"date": "2023-05-25T10:00:00.000Z"
}, {
"tid": 1,
"sq": 1,
"state": "assigned",
"date": "2023-05-25T11:00:00.000Z"
}, {
"tid": 1,
"sq": 2,
"state": "assigned",
"date": "2023-05-25T12:00:00.000Z"
}, {
"tid": 1,
"sq": 1,
"state": "assigned",
"date": "2023-05-25T12:00:00.000Z"
}, {
"tid": 1,
"sq": 2,
"state": "assigned",
"date": "2023-05-25T13:00:00.000Z"
}, {
"tid": 1,
"sq": 1,
"state": "assigned",
"date": "2023-05-25T14:00:00.000Z"
}, {
"tid": 1,
"sq": 1,
"state": "assigned",
"date": "2023-05-25T15:00:00.000Z"
}])



10.2)  Perform aggregation while using some options supported by aggregate function.


db.demo.aggregate([{
"$match": {
"state": "assigned",
"date": {
"$gte": "2023-05-25T10:00:00.000Z",
"$lte": "2023-05-26T10:00:00.000Z"
}
}
}, {
"$sort": {
"date": 1
}
}, {
"$group": {
"_id": {
"state": "$state",
"sq": "$sq",
"ticketId": "$tid"
},
"firstdate": {
"$first": "$date"
},
"lastdate": {
"$last": "$date"
},
"doc": {
"$first": "$$ROOT"
}
}
}])


10.3) Query result will be as follows: 

{
"_id": {
"state": "assigned",
"sq": 2,
"ticketId": 1
},
"firstdate": "2023-05-25T12:00:00.000Z",
"lastdate": "2023-05-25T13:00:00.000Z",
"doc": {
"_id": ObjectId("646fbf7fada80b4697d958a5"),
"tid": 1,
"sq": 2,
"state": "assigned",
"date": "2023-05-25T12:00:00.000Z"
}
} {
"_id": {
"state": "assigned",
"sq": 1,
"ticketId": 1
},
"firstdate": "2023-05-25T10:00:00.000Z",
"lastdate": "2023-05-25T15:00:00.000Z",
"doc": {
"_id": ObjectId("646fbf7fada80b4697d958a3"),
"tid": 1,
"sq": 1,
"state": "assigned",
"date": "2023-05-25T10:00:00.000Z"
}
}





9) aggregate pipeline 






Friday, 30 December 2022

Coding trends


Asynchronous code/ Non Blocking code

Asynchronous routine is able to wait while waiting on ultimate results to let other routines work in the meantime. 

Through this approach or mechanism asynchronous routine supports or achieve concurrency. 

Main beauty of this code is that it does this with single thread. 


Monday, 3 October 2022

Java 8 Features

1)Lambda expression/function: is an anonymous function that can be passed around where anonymous means that it does not have name .It is function as it is not associated with class like method. Passed around means it can passed as an argument to a method or can be stored in a variable.

Usage ::

Used to represent the instance of the functional interface.

syntax is as follows:

             (a,b)->System.out.println(a+b);

 Advantage ::

   No boiler plate code for simple things.

 

2)Functional interface: is an interface which contains only one abstract method but can have any number of default methods.

@FunctionalInteface

public interface Predicate{

boolean test(T t);

}


@FunctionalInterface

public interface Consumer {

void accept(T t);

}


@FunctionalInterface

public interface Function{

 public R apply(T t); 

}


@FunctionalInterface

public interface Supplier{

 public T get();

}


3)Default method inside Interface :

public interface MyInterface { // regular interface methods default void defaultMethod() { // default method implementation } }

In a typical design based on abstractions, where an interface has one or multiple implementations, if one or more methods are added to the interface, all the implementations will be forced to implement them too. Otherwise, the design will just break down.

Default interface methods are an efficient way to deal with this issue. They allow us to add new methods to an interface that are automatically available in the implementations. Therefore, we don't need to modify the implementing classes.

In this way, backward compatibility is neatly preserved without having to refactor the implementers.

what happens when a class implements several interfaces that define the same default methods.

In that case, the code simply won't compile, as there's a conflict caused by multiple interface inheritance (a.k.a the Diamond Problem). 

To solve this ambiguity, we must explicitly provide an implementation for the methods by overriding the implementation in class.


@Override public String turnAlarmOn() { return Vehicle.super.turnAlarmOn(); } @Override public String turnAlarmOff() { return Vehicle.super.turnAlarmOff(); }


4)static method inside Interface

5)Predicate<T> : has test method returns boolean

6)Function <T,R> : has apply method return R type after applying some business logic.

7)Consumer<T> : has accept method and it does not return anything instead can be used for iteration.

8)Supplier< > : has get Method and it returns  T type object.

9)Method reference and constructor reference by using :: (double colon ) operator.

10)Streams

11)Date & Time API (Joda API) 





Saturday, 1 October 2022

What is PermGen?

PermGen is memory area which was part of heap prior to Java 8. This is used to load class and method objects which means it was directly related to the number of classes objects being created. So when number of classes increases there objects can also increase and hence we use to face one issue java.lang.OutOfMemory error due to PermGen Size. We use to increase PermGen size using xx:MaxPermGen .


But there is new change in java 8 wherein this PermGen has been replaced by Metaspace which means there is no PermGen in java 8 onwards.

 Old JVM memory consist of  1)Heap  2)Native memory  

             wherein Heap consists of  

              a)Old generation.     b)New Generation.    c)PermGen

 New JVM memory consists of  1)Heap  2)Native memory

           wherein Heap consists of  

              a)Old generation.     b)New Generation

             Native memory consists of 

             a)Metaspace

Now metaspace is controlled by the native memory which is dependent of the host .So in case many classes are loaded due to xyz reason it may happen that process size can increase immensely and by so entire server can crash not only application.

So for this we have new parameter using which we can we can limit size of metaspace  given by xx::MaxMetaspaceSize. 


Conclusion:: 

1)we need to monitor heap as well as process size now. Process size can be monitored using system utilities like top in unix/linux and Task Manager in windows.

2)Jmap can be used as follows: Jmap -permstat <PID> 

3)No more PermGen space from java 8 onwards. Inclusion of metaspace.

3)QA should be made aware of this while doing testing.