Failed to deploy testwebapp

Hello experts,

I created WebLogic domain on pv and got error while deploying testwebapp. Below is information of steps: The WebLogic Operator version: v3.0.0

I used script in kubernetes\samples\scripts\create-weblogic-domain\domain-home-on-pv\create-domain.sh, the domain was created successfully with the follow output:

Creating weblogic domain domain1
Input parameters being used
export version="create-weblogic-sample-domain-inputs-v1"
export adminPort="7001"
export adminServerName="admin-server"
export domainUID="domain1"
export domainHome="/shared/domains/domain1"
export serverStartPolicy="IF_NEEDED"
export clusterName="cluster-1"
export configuredManagedServerCount="5"
export initialManagedServerReplicas="2"
export managedServerNameBase="managed-server"
export managedServerPort="8001"
export image="store/oracle/weblogic:12.2.1.3"
export imagePullPolicy="IfNotPresent"
export imagePullSecretName="haicheregcred"
export productionModeEnabled="true"
export weblogicCredentialsSecretName="domain1-weblogic-credentials"
export includeServerOutInPodLog="true"
export logHome="/shared/logs/domain1"
export httpAccessLogInLogHome="true"
export t3ChannelPort="30012"
export exposeAdminT3Channel="false"
export adminNodePort="30701"
export exposeAdminNodePort="true"
export namespace="default"
javaOptions=-Dweblogic.StdoutDebugEnabled=false
export persistentVolumeClaimName="haicheazurefile"
export domainPVMountPath="/shared"
export createDomainScriptsMountPath="/u01/weblogic"
export createDomainScriptName="create-domain-job.sh"
export createDomainFilesDir="wlst"
export serverPodMemoryRequest="768Mi"
export serverPodCpuRequest="250m"
export istioEnabled="false"
export istioReadinessPort="8888"

Generating /home/haiche/azure/weblogic-domains/domain1/create-domain-job.yaml
Generating /home/haiche/azure/weblogic-domains/domain1/delete-domain-job.yaml
Generating /home/haiche/azure/weblogic-domains/domain1/domain.yaml
Checking to see if the secret domain1-weblogic-credentials exists in namespace default
Checking if the persistent volume claim haicheazurefile in NameSpace default exists
The persistent volume claim haicheazurefile already exists in NameSpace default
W0810 09:51:07.364919    4841 helpers.go:535] --dry-run is deprecated and can be replaced with --dry-run=client.
configmap/domain1-create-weblogic-sample-domain-job-cm created
Checking the configmap domain1-create-weblogic-sample-domain-job-cm was created
configmap/domain1-create-weblogic-sample-domain-job-cm labeled
Checking if object type job with name domain1-create-weblogic-sample-domain-job exists
No resources found in default namespace.
Creating the domain by creating the job /home/haiche/azure/weblogic-domains/domain1/create-domain-job.yaml
job.batch/domain1-create-weblogic-sample-domain-job created
Waiting for the job to complete...
Error from server (BadRequest): container "create-weblogic-sample-domain-job" in pod "domain1-create-weblogic-sample-domain-job-j5bgz" is waiting to start: PodInitializing
status on iteration 1 of 20
pod domain1-create-weblogic-sample-domain-job-j5bgz status is Init:0/1
Error from server (BadRequest): container "create-weblogic-sample-domain-job" in pod "domain1-create-weblogic-sample-domain-job-j5bgz" is waiting to start: PodInitializing
status on iteration 2 of 20
pod domain1-create-weblogic-sample-domain-job-j5bgz status is Init:0/1
status on iteration 3 of 20
pod domain1-create-weblogic-sample-domain-job-j5bgz status is Running
status on iteration 4 of 20
pod domain1-create-weblogic-sample-domain-job-j5bgz status is Completed
domain.weblogic.oracle/domain1 created

Administration console access is available at http://haicheaksc-haicheresourcegr-685ba0-1eda3c35.hcp.eastus.azmk8s.io:30701/console
The following files were generated:
  /home/haiche/azure/weblogic-domains/domain1/create-domain-inputs.yaml
  /home/haiche/azure/weblogic-domains/domain1/create-domain-job.yaml
  /home/haiche/azure/weblogic-domains/domain1/domain.yaml

Completed

and then I create load balancer with the configuration:

Load balancer for admin server:

apiVersion: v1
kind: Service
metadata:
  name: domain1-admin-server-external-lb
  namespace: default
spec:
  ports:
  - name: default
    port: 7001
    protocol: TCP
    targetPort: 7001
  selector:
    weblogic.domainUID: domain1
    weblogic.serverName: admin-server
  sessionAffinity: None
  type: LoadBalancer

Load balancer for cluster:

apiVersion: v1
kind: Service
metadata:
  name: domain1-cluster-1-external-lb
  namespace: default
spec:
  ports:
  - name: default
    port: 8001
    protocol: TCP
    targetPort: 8001
  selector:
    weblogic.domainUID: domain1
    weblogic.clusterName: cluster-1
  sessionAffinity: None
  type: LoadBalancer

Then I deployed the testwebapp, and got the following error:

[partition-name: DOMAIN] > <BEA-149078> <Stack trace for message 149004
weblogic.management.DeploymentException: [Deployer:149189]An attempt was made to execute the "distribute" operation on an application named "testwebapp" that is not currently available. The application may have been created after non-dynamic configuration changes were activated. If so, the operation cannot be performed until the server is restarted so that the application will be available.
	at weblogic.deploy.internal.targetserver.DeploymentManager.assertDeploymentMBeanIsNonNull(DeploymentManager.java:1598)
	at weblogic.deploy.internal.targetserver.DeploymentManager.findDeploymentMBean(DeploymentManager.java:1643)
	at weblogic.deploy.internal.targetserver.DeploymentManager.createOperation(DeploymentManager.java:1172)
	at weblogic.deploy.internal.targetserver.DeploymentManager.createOperations(DeploymentManager.java:1690)
	at weblogic.deploy.internal.targetserver.DeploymentManager.handleUpdateDeploymentContext(DeploymentManager.java:182)
	at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.updateDeploymentContext(DeploymentServiceDispatcher.java:173)
	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doUpdateDeploymentContextCallback(DeploymentReceiverCallbackDeliverer.java:155)
	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.updateDeploymentContext(DeploymentReceiverCallbackDeliverer.java:29)
	at weblogic.deploy.service.internal.statemachines.targetserver.ReceivedPrepare.callDeploymentReceivers(ReceivedPrepare.java:208)
	at weblogic.deploy.service.internal.statemachines.targetserver.ReceivedPrepare.handlePrepare(ReceivedPrepare.java:112)
	at weblogic.deploy.service.internal.statemachines.targetserver.ReceivedPrepare.receivedPrepare(ReceivedPrepare.java:52)
	at weblogic.deploy.service.internal.targetserver.TargetRequestImpl.run(TargetRequestImpl.java:214)
	at weblogic.deploy.service.internal.transport.CommonMessageReceiver$1.run(CommonMessageReceiver.java:614)
	at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:670)
	at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)
	at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)
	at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)
	at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)
	at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:644)
	at weblogic.work.ExecuteThread.execute(ExecuteThread.java:415)
	at weblogic.work.ExecuteThread.run(ExecuteThread.java:355)
> 

could you help to check? Thank you

[jshum2479]

Hi @galiacheng ,

Are you able to make further progress in this issue or is it still an issue?

I think the main difference between my environment and your environment is the PV, other than I use a specific resource group

1. I added the persistentVolumeReclaimPolicy: Retain
2. Removed the mountOptions
3. I pointed the right testwebapp.war to use, but you can compare the md5sum between the one in src/integration-tests/apps and yours

[galiacheng]

This issue is solved by applying pv/pvc config from @jshum2479. Thanks for your help @jshum2479

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolume
metadata:
  name: fileshare-pv
  labels:
    usage: fileshare-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  storageClassName: azurefile
  persistentVolumeReclaimPolicy: Retain
  azureFile:
    secretName: azure-secret
    shareName: fileshare1
    readOnly: false
  mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1000
  - gid=1000
  - mfsymlinks
EOF

cat <<EOF | kubectl apply -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: fileshare-pvc
spec:
  storageClassName: azurefile
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  selector:
    matchLabels:
      usage: fileshare-pv
EOF